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ABSTRACT 

Slmulatlona  of  oompntar  ayatasu  hava  traditionally  bean  parformed  on  a 
slnsla,  aofuantlal  computer,  afvan  If  the  as^atem  to  be  almulatad  oontalna  a 
number  of  oomponenta  which  operate  concurrently.  An  alternative  would  be  to 
almulate  thaaa  ayatema  on  a network  of  proceaaora.  With  thia  approach,  each 
processor  would  almulate  one  component  of  the  ayatem,  hence  the  component 
aimulationa  could  proceed  poncurrantly.  By  exploitiaA  the  modularity  and 
concurrency  la  the  ayatem  to  be  almulated,  the  almulatlon  would  itself  be 
modular  and  concurrent. 

An  accurate  simulation  must  model  the  time  behavior  of  the  system  as 
well  as  its  input-output  behavior.  In  order  to  avoid  real-time  constraints  on 
the  processors  and  communication  network  in  the  stmuiaition  facility,  the 
simulation  of  the  timing  must  use  a time-independent  algorithm.  That  is,  the 
almulated  behavior  of  each  component  should  not  depend  on  the  speed  at  which 
the  simulation  la  performed. 

With  this  tlme-indepmidont  approach,  additional  coordination  operations  are 
required  to  prevent  a deadlock  of  the  simulation.  This  coordination  be 
provided  without  any  centralized  control.  Instead,  the  program  for  the 
simulation  of  each  component  la  modified,  so  that  each  component  simulation 
will  communicate  status  information  to  other  component  simulations.  Additional 
tormination  operations  are  also  required  to  assure  that  the  simulation  will 
terminate  under  the  exact  same  conditions  that  the  system  beind 
would  tannlnate.  These  operations  can  also  be  provided  without  any 
centralization  of  control  or  real-time  constraints.  Furthermore  a simulation 
which  uses  these  coordination  and  termination  operations  is  provably  correct. 
That  is,  the  simulation  will  accurately  model  both  the  time  behavior  and  the 
input-output  bMiavlor  of  the  system. 
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Chapter  1 ! 

Introdaotion  { 

/ 

I 

1 

Computer  Systems  have  traditionally  been  simulated  on  a sin^e,  sequential  | 

computer,  even  if  the  system  to  be  simulated  contains  a number  of  components 

i 

which  operate  In  parallel.  One  of  the  primary  purposes  of  simulation  languages, 
such  as  OPSS  and  Simscrlpt  n [13],  is  to  order  the  simulation  of  the  events 
occurln^  lu  the  different  components  in  such  a way  that  the  simulation  will 
correctly  model  the  operation  of  the  system  to  be  simulated.  An  alternative 
approach  would  be  to  simulate  parallel  systems  on  a network  of  computers,  such 
as  a network  of  microprocessors  [2,14,21]  or  the  Arpanet  [IS],  where  each 
processor  would  simulate  the  operations  of  one  component  of  the  system.  This 
would  allow  the  simulation  to  exploit  the  modularity  and  concurrency  of  the 
system  to  be  simulated  and  thereby  itself  achieve  a high  level  of  modularity 
and  concurrency.  The  simulation  of  packet  communication  architecture  systems 

I 

[6]  seems  particularly  suited  for  this  approach,  since  these  ss^ms  are  highly 
modular  - the  components  of  the  system  operate  independently  and  communicate 
with  each  other  only  by  sending  message  packets.  Hence  these  systems  can  be 

i 

simulated  by  a network  of  processors  which  communicate  by  message  passing. 

1 

I 

Pmekmt  Comnranionilon  Arohitnotor* 

A packet  communication  architecture  system  consists  of  a number  of 
independent  processor  modules  which  communicate  by  sending  packets  of  j 

information  to  one  another.  A single  program  is  implemented  as  a number  of  I 

separata  processes,  such  that  each  process  runs  on  one  of  the  modules,  hence  the  I 
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compoiients  of  tha  profnai  can  ba  azacutad  In  parallaL 


Tha  Tn«*d"l—  In  a paCkat  oonunnnlcation  archltactura  lyitam  can 
communlcata  only  In  a Umltad  fashion.  All  communication  with  a module  is  in 
tha  form  of  packats,  except  the  initial  state  of  the  module,  which  can  ha  given 
to  the  module  in  nonpackat  form.  Thus,  a module  could  he  initialized  with  a 
program  and  data,  hut  thereafter  it  can  receive  information  only  in 

packets.  Furthermore,  a module  can  communicate  with  only  a limited  number 
of  other  module*.  Eadi  module  receives  and  sends  out  packets  through  its 
input  and  output  ports.  A particular  input  port  to  a module  can  receive  packets 
only  from  a particular  output  port  of  some  module,  or  ftom  a particular  source 
outside  tha  system.  Input  ports  of  the  latter  type  are  called  system  input  ports. 


since  they  are  the  only 


for  an  external  source  to  send  data  to  the  system. 


Similarly,  from  a partlculmr  output  port  of  a module,  packets  can  he  sent  only 
to  a particular  input  pent  of  some  module  or  to  a particular  external  destination. 
Output  porta  from  which  packets  ate  sent  to  external  destinations  are  caUed 
system  output  ports. 


Packets  are  carrlad  along  one-way  data  channels  from  the  output  port  of 


one  module  to  the  input  pent  of  another.  These  channels  cannot  altar  tha  values 
of  the  packets,  and  they  must  prasarve  the  sequential  ordering  of  tha  packets. 
Thus,  a can  ho  viewed  as  a FIFO  queue  between  two  ports.  The 

intarconnactlons  between  modules  cannot  he  changed  dynamically. 


The  mtrlultf  in  a packet  architecture  system  operate 
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autonomously.  There  Is  no  central  control  In  the  system,  and  any  monitoring;  of 
the  system  operation  must  be  passive.  That  Is,  only  an  external  observer  Is 
allowed  to  monitor  the  modules  or  channels  In  the  system,  and  the  monitoring 
Is  not  vital  to  the  system's  correct  operation.  As  a result  of  this  autonlmlty,  a 
module  can  operate  as  soon  as  the  necessary  data  packets  have  arrived  regardless 
of  the  status  of  other  modules  In  the  system. 

A packet  communication  architecture  system  Is  designed  so  no  component 
of  the  system  will  be  required  to  fulfill  any  timing  constraints.  Instead,  the 
system  must  be  designed  to  operate  correctly  regardless  of  the  delay  times  or 
throughputs  of  the  modules  and  channels.  For  example,  one  module  cannot 
require  another  module  to  have  a laiwimtim  response  time.  As  a result,  modules 
must  use  aeynchronous  communication  protocols,  so  that  a module  cannot  send  a 
data  value  to  another  module  which  lacks  sufficient  buffer  space.  This 
communication  protocol,  however,  must  be  Implemented  as  packets  sent  back  and 
forth  between  two  modules  for  each  data  transfer.  Otherwise,  an 
acknowledgement  signal  received  from  a module  to  which  data  has  been  sent 
would  constitute  a form  of  nonpacket  input  information. 

As  a consequence  of  this  time-independent  design,  the  speed  of  the  system 
or  any  of  its  components  Is  a performance  Issue  and  not  a necessary  requirement 
for  correct  operation.  If  one  module  or  channel  Is  particularly  slow.  It  might 
slow  down  the  entire  system,  but  it  will  not  cause  any  malfunctions. 

Examples  of  pecket  communication  architecture  systenm  Include  the  data 


) 


f 
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flow  procossors  of  Oonals  and  Mlsunu  [7,8,9]  and  the  data  flow  processor  of 
Rtunbaugh  [2.0].  While  not  precisely  a packet  communication  architecture 
ssrstem  (due  to  dynamically  changeable  Interconnections)  the  Distributed 
Computing  System  at  the  University  of  California,  Irvine,  when  running  with 
the  DCOS  operating  system  [19],  embodies  many  of  the  same  design  philosophies. 

Advantages  of  Psudeet  Oommunioation  Arohiteoture  Systems 

These  systems  have  several  major  advantages  over  both  traditional  computer 
systems  and  other  designs  for  parallel  systems.  First,  the  modules  In  the  system 
can  operate  concurrently,  thereby  achieving  a high  rate  of  computation.  Since 
there  Is  no  central  control,  there  Is  no  component  which  will  Inherently  cause 
a bottleneck  In  Uie  system,  or  which  must  have  an  extremely  high  throughput 
in  order  to  keep  the  rest  of  the  system  operating  at  a reasonable  rate. 

Second,  the  system  can  be  designed  modularly,  by  first  speclfsrlng  the 
functional  requirement  for  each  module  as  well  as  some  connection  standards 
and  then  designing  the  modules  Individually.  Since  modules  can  Interact  only  in 
limited  and  wellHleflned  ways,  as  opposed  to  systems  which  contain  shared 
memories  or  allow  Interrupts,  for  example,  a module  has  a very  clean  interface 
with  the  rest  of  the  system.  Furthermore,  since  there  are  no  timing  restrictions 
on  a module,  the  specifications  for  Its  operation  need  contain  only  Its  functional 
operation,  l.e.  the  output  packets  produced  In  response  to  a set  of  Input  packets. 

Once  a system  has  been  designed,  we  can  try  to  maximize  Its  performance. 
This  Involves  Identifying  the  modules  and  channels  In  the  system  which  are 


consistently  heavily  loaded  and  hence  form  bottlenecks  In  the  system.  A 
bottleneck  can  be  eliminated  by  redesigning  the  module  or  channel  to  operate 
faster  or  by  splitting  one  module  Into  several  modules.  Because  the  system  Is 
designed  to  be  speed  Independent,  the  speed  of  one  module  can  be  varied 
without  causing  malfunctions. 

One  further  result  of  this  modularity  of  design  Is  that  these  systems  can 
be  proved  correct  much  more  easily  than  other  computer  systems.  To  prove  the 
correctness  of  a packet  communication  architecture  system,  one  can  specify  the 
required  properties  of  each  module,  prove  that  each  module  satisfies  these 
properties,  and  then  prove  that  the  sjrstem  will  operate  correctly  If  all  modules 
satisfy  their  requirements.  In  other  words,  the  correctness  of  the  system  can  be 
proved  modularly.  General  methods  of  proving  the  correctness  of  packet 
communication  architecture  systems  are  currently  being  Investigated  by  Ellis 
[10]. 

Examples  of  Packet  Communioation  Arokiteoture  Modules 

Three  basic  module  t}rpes:  functional  operators,  switches,  and  arbiters 
Illustrate  some  of  the  operations  which  can  be  performed  by  packet 
communication  architecture  modules.  Examples  of  their  operation  are  shown  in 
Figure  1.1.  In  the  diagrams  the  lines  represent  the  channels  connected  to  the 
Input  and  output  ports  of  the  modules,  and  the  dots  on  these  lines  represent 
data  packets  being  transmitted  over  the  channels. 


A functional  operator  computes  several  functions  (one  for  each  output  port) 
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with  Input  packets  as  arguments.  It  can  fire  as  soon  as  one  packet  Is  received 
at  each  Input  port,  meaning  that  It  absorta  these  input  packets,  computes  the 
output  values,  and  sends  one  output  packet  from  each  output  port.  For 
example,  the  DIVIDE  module  of  Figure  1.1a  computes  two  functionst  the 
quotient  and  the  remainder  of  the  input  values. 

A switch  module  provides  a means  of  routing  data  to  different  modules  in 
the  system.  It  can  fire  as  soon  as  a packet  is  received  on  its  input  port.  In 
firing,  it  absorbs  the  input  packet  and  then  sends  an  identical  output  packet 
from  one  ot  several  output  ports,  depending  on  the  packet's  value.  In  the 
example  of  Figure  1.1b,  the  output  port  selected  depends  on  whether  the  packet 
value  is  positive,  zero,  or  negative. 

As  a final  example,  the  arbiter  module  serves  to  merge  together  the 
streams  of  output  packets  from  several  modules.  It  can  fire  as  soon  as  a packet 
is  received  on  either  input  port.  In  firing,  it  absorbs  a packet  from  one  of  the 
input  ports  and  sends  an  identical  packet  from  its  output  port.  If  packets  are 
received  at  two  input  ports  simultaneously,  the  module  will  first  fire,  absorb 
one  of  these  packets,  and  send  it  out.  By  the  rules  of  operation,  any  packet 
which  is  not  absorbed  will  remain  at  the  input  port.  Hence,  the  module  will 
fire  a second  time,  absorb  the  remaining  packet,  and  send  this  one  out. 


Other  packet  communication  architecture  modules  can  have  behaviors 
which  depend  on  other  factors,  such  as  past  activities  of  the  module,  the  arrival 
times  of  the  input  packets,  and  stochastic  processes  within  the  module.  The 
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general  rules  of  operation  for  the  modules  will  be  discussed  In  Chapter  2. 

The  Need  for  Simuletlon 

Once  the  functional  behavior  of  all  components  have  been  developed  and 
proved  correct,  there  are  other  Issues  to  be  settled  before  the  system  can  be 
Implemented.  The  Implementation  must  meet  other  requirements  on  the  overall 
speed  of  operations  or  the  total  cost  of  the  system.  Thus,  for  a particular 
Implementation,  a designer  will  want  to  measure  the  performance  of  the  system 
for  different  sets  of  Input  data.  These  measurements  can  include  such  factors 
as  the  overall  ^eed  of  the  system,  the  load  on  particular  components,  and  the 
buffering  requirements  at  the  Input  ports.  Once  measurements  for  a particular 
Implementation  have  been  made,  the  designer  will  want  to  make  measurements 
when  such  parameters  as  throughput  or  delay  time  for  particular  components 
have  been  varied,  or  modifications  have  been  made  to  the  original  design.  By 
this  method,  the  designer  can  tna-rimiMt  the  speed  and  mlnlmlaw  the  cost  of  the 
system. 

Measurements  of  a system's  performance  are  required  not  only  to  find  an 
optimum  implementation,  but  also  to  compare  the  system  to  other  system 
designs,  or  to  conventional  computer  systems.  While  packet  communication 
architecture  systems  are  potentially  very  fast  due  to  the  high  level  of 
parallelism,  a method  of  comparison  with  traditional  computer  systems  is 
desired. 


Developing  mathematical  methods  of  predicting  the  performance  of 


particular  systems  seems  to  be  very  difficult.  One  cannot  simply  count  the 
number  of  instruction  cycles  required  for  a particular  program  with  a particular 
set  of  Input  data.  While  the  modules  Interact  with  each  other  In  a very 
limited  and  well-defined  way  from  a functionality  viewpoint,  the  performance 
of  a module  can  have  very  subtle  effects  on  the  performance  of  the  overall 
ssrstem.  For  example.  Increasing  the  throughput  of  one  module  can  cause 
another  module  to  become  a bottleneck  in  the  system.  Thus,  a "modular" 
approach  to  performance  analysis  will  not  work.  Furthermore,  the  system 
designer  wants  to  know  more  than  Just  the  average  or  worst  case  performance 
of  some  sj'stem.  He  wants  to  know  the  detailed  performance  measurements  for 
each  componeint  of  the  system.  This  amount  of  detail  could  never  be  provided 
accurately  by  a mathematical  analysis  of  performance. 

An  accurate  simulation  of  a system  would  provide  the  desired 
measurements  for  a particular  set  of  Input  data.  While  it  might  be  hard  to 
Judge  the  general  performance  of  a system  based  on  simulations  for  a few  sets 
of  Input  data,  this  approach  seems  to  provide  a great  deal  more  Information  than 
analytic  methods. 

To  avoid  confusion  between  the  ssrstem  to  be  simulated  and  the  system 
which  performs  the  simulation,  the  former  will  be  called  the  actual  system,  and 
the  latter  will  be  called  the  simulation  sy8teii\.  Even  though  the  "actual" 
system  might  In  fact  only  exist  on  paper,  this  seems  like  a reasonable  way  to 
distinguish  the  two.  Furthermore,  the  modules  and  channels  of  the  actual 
system  will  be  called  the  actual  modules  and  actual  channels. 


Requiremanta  for  tha  Bimolatloii 

To  provldo  tho  typ*  of  niMsnreiiMiitf  roqulnd  to  evaluate  an 
Implementation  of  a system,  the  simulation  must  accurately  model  all  aspects  of 
the  system's  operations.  This  Includes  modeUlnf  the  detailed  timing  aspects  of 
the  system  as  well  as  the  functional  behavior.  If  only  the  functional  aspects 
were  modelled,  the  simulation  would  accurately  model  some  Implementation  of 
the  system,  but  most  likely  not  the  implementation  we  are  interested  In. 

An  accurate  modelllnA  of  the  system  cannot  rely  on  any  stochastic  methods 
of  simulation,  unless  the  modules  themselves  behave  stochastically.  For  one 
thlnA,  Uke  analytic  methods,  methods  of  stochastically  modelllnA  packet 
communication  architecture  systems  have  not  yet  been  developed.  Thus, 
the  ssrstem  is  affected  by  stochastic  processes  within  the  modules,  a simulation 
of  a system  should  provide  all  Information  about  the  activities  of  each  module 
for  a given  set  of  Initial  states  (l.e.  module  programs  and  Initial  data),  and  a 
particular  sequence  of  Input  packets  presented  to  each  system  Input  port.  If  the 
modules  behave  stochastically,  the  stochastic  processes  must  be  modelled,  so  that 
any  random  variables  will  be  chosen  with  the  same  probability  In  the 
simulation  as  they  are  In  the  actual  system.  A single  simulation  will  only 
model  the  system's  activity  for  one  choice  of  random  variables,  but  a number  of 
simulations  can  give  an  idea  of  the  distribution  of  the  system's  performance. 

Methods  of  Slmoletlon 

One  approach  to  the  simulation  of  a packet  communication  architecture 
system  Is  with  a sequential  computer  system.  With  this  approach,  a single 


computer  would  simulate  the  activities  of  every  module  and  every 

communication  channel  In  the  system.  While  tnia  approach  wmuld  he  rather 

slow.  It  is  not  difficult  to  Implement.  For  every  packet  on  an  input  port  of 

some  module  in  the  ssrstem,  the  simulation  keeps  a packet  descriptor  of  the 

form  where 

M ■ the  module  number 
p • the  input  port  ntunber 
s • the  value  contained  in  the  packet 
t ■ the  time  at  which  the  packet  arrived  at  the  input  port. 

These  packet  descriptors  are  stored  as  a sequential  list  called  a time  line,  in 

which  the  descriptors  are  ordered  by  their  time  values.  The  simulation  looks  at 

the  time  line  and  decides  which  module  in  the  system  would  fire  the  soonest. 

It  then  simulates  the  firint  of  this  module  by  removing  the  absorbed  input 

packets  from  the  time  line,  computing  the  output  values  and  delay  time  for  the 

module,  and  then  inserting  new  packet  descriptors  for  each  output  packet  into 

the  time  line.  Each  new  packet  descriptor  contains  the  module  and  input  port 

number  of  the  input  port  which  receives  the  packet,  the  value  of  this  packet, 

and  the  time  at  which  the  input  port  would  receive  the  packet.  This  process  is 

repeated  for  the  new  time  line,  and  so  on,  until  no  module  in  the  system  is 

able  to  fire.  As  long  as  the  simulation  always  simulates  the  earliest  firing  in 

the  system  for  a divan  state  of  the  time  line,  it  can  be  certain  that  all  input 

packets  which  would  have  been  received  by  this  module  at  firind  time  are 

present  on  the  time  line.  Since  a module  cannot  be  affected  by  new  input 

packats  arrivind  while  it  is  flrind,  the  entire  firind  of  the  module  can  be 

simulated  without  looklnd  at  other  modules  in  the  system.  Simulation 


lan^uagw,  sucli  as  QPSS  aad  Slaucilpt  n [13],  um  a variant  of  this  tima  Una  in 
atmniating  tlM  activltias  Of  a numbar  of  ooncnrrant  procawea  on  a slntf a 
oomputar. 

A larga  fraction  of  tlia  fimnlation  time  will  be  spent  looking  at  the  time 
Una  to  dodda  which  module  would  fire  earliest.  Whereas  it  is  not  difficult  to 
determine  whather  simple  modules,  such  as  functional  operators,  switches,  or 
arbiters  are  ready  to  fire  and  at  what  time,  these  computations  could  take  much 
longer  fw  modules  with  more  complex  behavior.  Moreover,  as  the  size  of  the 
system  increases,  there  will  be  more  modules  to  check,  and  more  descriptors  on 
the  time  line.  Henoe,  the  time  spent  on  overhead  in  the  simulaUon  can,  in  the 
worst  case,  increase  as  the  siuare  of  the  system  sissi  there  wlU  be  a linear 
Increase  in  the  total  number  of  firings  to  be  simulated,  and  for  each  firing  a 
linear  Increase  in  the  time  required  to  decide  which  module  would  fire  earUest. 
The  time  spent  to  actually  simulate  the  activities  of  the  modules,  on  the  other 
hand,  wlU  increase  only  linearly  with  the  system  size.  As  the  size  of  the 
system  is  increased,  the  proportion  of  slmulatton  time  spent  on  overhead  wUl 
increase. 

An  aitematlve  to  simulation  on  a sequential  computer  is  to  simulate  the 
system  on  a computer  ■jsiem  of  a number  of  interconnected 

simulation  proeessoM,  sudi  as  the  Fackst  Architecture  Simulation  Facility  of 
LeunA,  at  ai  [14],  shovm  in  Flsura  1.2.  In  this  facility  microprocessors  serve 
as  simulation  processors.  Each  simulation  processor  simulates  one  or,  for  a large 
system,  several  of  the  modules  in  the  system.  The  processors  send  packets  to 


OM  anothar,  jturt  at  tha  modules  In  the  actual  system  would.  The  packets  are 
sent  over  a communication  network,  which  provides  connections  amonf  all  pairs 
of  simulation  processors.  Durlnf  a simulation,  however,  a processor  would  send 
packets  to  another  processor  only  If  the  first  is  MimnUtimi  • module  which  can 
send  packets  to  a module  beln^  simulated  by  the  second.  The  communication 
network  Is  provided  to  allow  the  simulation  of  any  system  configuration.  In 
addition,  a host  computer  can  load  programs  into  the  modules,  initiate  the 
simulation,  and  monitor  its  progress. 


Coeaunicatlon  Network 


Proceseor 


Proceaeor 


Hoat  Interface  Bua 


Figure  1.2  ~ Structure  of  Simulation  Facility 


This  approach  seems  very  natural,  since  the  structure  of  the  simulation  is 


much  Ilk*  that  of  th*  «yst*ai  Iwlii^  simulated.  It  should  also  be  faster,  since 
the  simulation  proceasors  can  operate  In  parallel.  Hopefully,  the  amount  of 

I 

overhead  will  not  be  too  great,  either,  so  that  a large  fraction  of  processor  time 
can  be  spent  simulating  the  activltlas  of  the  modules. 

Purpose  ot  Thesis 

In  this  thesis,  methods  of  simulating  packet  communication  architecture 
systems  on  a distributed  computer  system  will  be  developed.  The  design  goals 
for  theaa  simulation  methods  ineludet 


1.)  Generality  ot  SimuIaUng  System.  The  simulation  should  not 
require  a highly  specialized  computer  system  on  which  to  perform 
th*  slmulaton.  It  should  work  on  any  system  which  supports 
communicating  processes,  such  as  the  Packet  Architecture  Simulation 
Facility  [14],  a network  of  microprocessors  [2,Z1],  the  Distributed 
Computing  System  [11,19],  or  even  more  traditional  systems  such  as 
the  Burroughs  86700  [16]. 


2.)  Generality  ot  Simulation.  The  methods  should  enable  the 
accurate  simulation  of  any  packet  communication  architecture 
system.  A system  designer  should  not  be  limited  in  the  types  of 
systems  which  he  can  simulate. 


3).  Simplicity  ot  Software.  The  programs  for  each  simulation 
processor  should  be  reasonably  simple  to  write,  and  short  enough  to 
be  executed  by  small  processors  such  as  microprocessors. 


4).  Reasonable  Rtticiencyx  The  simulation  should  make  use  of  the 
potential  parallelism  In  the  simulation  system.  Furthermore,  the 
amount  of  communication  between  processors  to  keep  their  efforts 
coordinated  should  be  reasonably  small. 

One  way  to  satisfy  the  first  goal  would  be  for  th*  simulation  itself  to  have  the 
properties  of  a packet  communication  architecture  system.  First,  the  simulation 
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processors  should  act  autonomously,  with  no  central  control.  This  will  simplify 

the  computer  system  required  to  perform  the  simulation  by  removing  the  need 

for  a highly  specialized,  high  speed  central  controller.  Of  course,  passive  j 

monitoring  might  be  allowed  to  observe  the  simulation  activities.  Second,  all 

communication  between  simulation  processors  should  be  In  the  form  of  packets. 

As  a result,  the  processors  will  have  a uniform  form  of  input-output.  Perhaps 
most  importantly,  the  simulation  will  be  time-independent.  That  Is,  the 
accuracy  and  correctness  of  the  simulation  will  not  depend  on  the  speed  of  the 
simulation  processors  or  the  communication  network.  This  will  eliminate  any 
real  time  constraints  on  the  simulation  hardware  and  software,  which  will  ^ 

i 

greatly  simplify  the  design.  This  will  also  enable  the  simulation  to  be  ! 

i 

performed  on  any  computer  system  which  supports  gommiintr^ttng  processes.  4 

The  simulation  of  each  component  of  a system  could  be  handled  by  a different 
process.  Several  of  these  processes  could  be  assigned  to  one  processor,  which 
could  execute  them  without  any  time  constraints. 

While  the  simulation  might  be  faster  on  a highly  specialized  simulation 
facility  equipped  with  a high  speed  controller  or  processors  designed  for  real 
time  applications,  the  amount  of  time  and  money  required  to  construct  such  a 
facility  would  be  Justified  only  If  a very  large  number  of  simulations  were  to 
be  performed. 


The  problem  then  becomes  developing  simulation  methods  'based  on  packet 
communication  architecture  principles,  which  will  satisfy  the  other  three  goalst 
generality,  simplicity  of  software,  and  reasonable  efficiency.  One  means  of 
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simpUfyliif  th«  twlc  of  flofUm*  diign  Is  to  tdu  a modular  approach  to  the 
design  of  simulation  programs.  The  simulation  program  for  a module  must  not 


only  simulate  the  aetttrttles  of  the  module,  it  most  also  communicate  with  other 
module  programs  to  keep  the  simulation  activities  coordinated.  Thus,  the 
specifications  for  each  simulation  program  will  include  not  only  specifications  of 
the  module  to  be  simulated,  but  also  specifications  of  the  coordination  activities. 
To  keep  the  design  modular,  the  coordination  activities  must  be  simple  and 
uniform  enough  to  be  easily  and  accurately  specified.  Moreover,  these 
coordination  activities  must  be  both  general  and  reasonably  efficient.  The  major 
task  of  this  thesis  is  to  develop  coordination  methods  which  fulfill  the 


requirements  of  simplicity,  generality,  and  efficiency  for  a simulation  which  is 
Itself  a packet  commanlcatlon  architecture  system. 

Outline  of  Theele 

In  Chapter  2 methods  of  simulating  the  components  of  a packet 
communication  architecture  system,  i.e.  the  modules  and  communication 
channels,  will  be  discussed.  First,  rules  of  operation  for  packet  communication 
architecture  modules  will  be  presented.  Then,  methods  of  simulating  both  the 
functional  and  timing  aspects  of  the  module  will  be  developed.  The  emphasis 
will  be  on  speclfsrlng  what  a correct  simulation  of  a module  would  do,  rather 
than  on  the  more  difficult  problem  of  translating  these  requirements  into  actual 
programs.  The  problem  of  producing  programs  which  will  accurately  simulate  a 


module,  based 


specification  of  the  module,  is  left  as  an  area  for  further 


In  Chapter  3 the  ideas  developed  in  Chapter  2 will  be  ej^nded  to  allow 
the  simulation  of  entire  systems.  As  will  be  seen,  if  the  simulation  processors 
are  simply  loaded  with  programs  which  simulate  the  activities  of  the  system 
components,  the  slmxQation  might  not  accurately  model  the  system  but  instead 
reach  a deadlocked  state.  Besides  simulating  the  activities  of  the  modules,  the 
simulation  processors  must  communicate  with  each  other  to  keep  their  efforts 
coordinated.  The  main  purpose  of  this  chapter  is  to  develop  methods  of 
incorporating  the  coordination  activities  into  the  simulation  processor  programs. 
In  this  chapter  a proof  will  be  described  which  shows  that  the  simulation  will 
accurately  model  the  actual  system.  The  full  proof  is  contained  in  Appendix  1. 
This  proof  demonstrates  the  benefits  of  the  modular  approach  to  the  design  of 
the  simulation.  First,  the  important  requirements  for  the  modules  in  the  system 
and  for  the  simulation  programs  of  these  modules  will  be  specified.  Second,  it 
will  be  proved  that  the  simulation  and  coordination  methods  of  Chapters  2 and 
3 satisfy  these  requirements.  Finally,  it  will  be  proved  that  any  simulation 
which  satisfies  the  requirements  will  accurately  model  the  actual  system. 

In  Chapter  4 methods  of  terminating  the  coordination  activities,  once  the 
modules  in  the  system  have  ceased  activity  will  presented.  Without  this 
termination,  the  simulation  might  run  indefinitely,  even  though  no  module 
activities  are  being  simulated.  The  last  part  of  the  chapter  describes  a proof  of 
the  correctness  of  the  termination  operations.  The  full  proof  is  contained  In 
Appendix  2.  First,  it  is  proved  that  these  operations  wlU  not  terminate  the 
simulation  too  soon  or  in  any  other  way  Interfere  with  the  simulation 
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operations.  Hence,  the  requirements  for  the  correctness  of  simulation  proof  will 
still  apply.  Then,  It  will  be  proved  that  the  simulation  wlU  eventually 
terminate.  If  the  actu^  system  would  terminate  under  the  same  circumstances. 

In  Chapter  5,  the  coordination  methods  of  Chapter  3 will  be  further 
refined  to  Increase  the  efficiency  of  the  simulation.  The  coordination  methods 
of  Chapter  3 are  designed  to  be  very  simple  and  uniform  over  all  modules.  As 
a result,  the  amount  of  coordination  Information  passed  between  processors  Is 
high,  and  the  concurrency  of  the  processors'  activities  can  be  unnecessarily 
restricted.  In  some  cases,  the  processor  program  for  a module  can  be  modified 
slightly  to  take  advantage  of  specific  properties  of  the  module.  Two  examples 
of  such  modifications  are  presented.  These  two  modifications  will  not  Increase 
the  complexity  or  modularity  of  the  simulation  programs  significantly  but  can 
greatly  Increase  the  efficiency  of  the  simulation.  Moreover,  these  modifications 
will  not  cause  the  simulation  programs  to  violate  any  of  the  requirements  for 
the  correctness  proof  of  Appendix  1 to  apply.  This  further  demonstrates  the 
benefits  of  a modular  approach  to  correctness  proofs. 

Finally,  Chapter  6 contains  conclusions,  suggestions  for  other  applications, 
and  suggestions  for  further  research.  Some  of  the  other  applications  Include 
simulation  of  other  types  of  sjrstams,  and  application  of  the  coordination  and 
termination  methods  to  other  forms  of  distributed  computation. 

By  working  within  the  concepts  of  packet  communication  architecture,  this 
thesis  develops  simulation  techniques  which  fulfill  the  four  design  goalsi 
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(implicity  of  hardware,  c^nerallty,  simplicity  of  software,  and  reasonable 
efficiency.  Moreover,  these  techniques  are  provably  correct.  This  Is 
particularly  comforting  considering  the  sultle  nature  of  parallel,  asynchronous 
computations,  which  can  often  have  unexpected  deadlocks,  races,  nontermination 
problems,  or  other  malfunctions. 


For  any  computation  which  is  designed  to  be  executed  by  a parallel, 
asynchronous  system  such  as  a packet  communication  architecture  system,  a 
proof  of  correctness  Is  essential.  The  traditional  approach  of  Implementing  an 
Initial  version  of  a system  and  then  debugging  It  will  not  work  for 
computations  which  must  be  time-independent.  Even  If  the  computation  Is 
tasted  on  a large  number  of  test  cases,  one  cannot  be  certain  that  It  will  be 
correct  for  all  cases.  A slight  change  in  the  timing  of  one  part  of  the 
computation  might  lead  to  a deadlock,  critical  race,  or  other  malfunction.  Even 
in  trying  to  prove  the  correctness,  one  can  easily  overlook  some  of  the 
‘ subtleties  of  the  computation.  However,  by  carefully  developing  a formal 

mathematical  description  of  the  computation  and  then  proving  that  a j 

computation  which  fulfills  this  description  will  operate  correctly,  these 

subtleties  can  be  uncovered.  i 
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Oh»pt«r  3 

EUmalatIng  th«  Components  of  n 
Paoket  Conunanioatlon  Arohiteoiore  System 

Introdnotlon 

Each  procosaor  is  tha  stmnlatioii  must  simulate  the  operations  of  one  or 
more  of  the  modules  or  communication  channels  In  the  actual  ssrstem.  This 
Includes  simulating  the  timing  details  of  the  module  as  well  as  the  module's 
data  operations.  If  the  simulation  Is  to  Itsrtf  be  a packet  communication 
architecture  system,  there  can  be  no  timing  constraints  on  the  simulation 
processors  or  on  the  communication  links  between  processors.  Hence,  a method 
of  simulating  the  timing  must  be  developed  which  Is  Independent  of  the  speed 
of  simulation. 

Modulo  Oporutlon 

Before  methods  of  simulating  modules  can  be  developed,  the  behavior 
which  will  be  expected  of  these  modules  must  be  presented.  In  the  Interest  of 
generality,  these  rules  will  be  as  unrestrictlve  as  possible.  As  a result,  some 
forms  of  behavior  are  allowed  which  are  not  quite  In  keeping  with  the 
philosophies  of  packet  communication  architecture  design.  However,  as 
mentioned  before,  the  designer  of  a system  should  not  be  restricted  In  the  types 
of  systems  he  can  simulate.  Furthermore,  these  allowances  do  not  cause  any 

e 

added  difficulties  for  the  simulation. 


At  any  time,  a module  is  in  one  of  two  modest  the  wait  mode  or  the  firing 


mode.  While  in  the  wait  mode,  the  module  cannot  produce  any  output  pw^heta. 
Once  the  necessary  conditions  for  firing  are  met,  the  module  fires,  that 

it  absorbs  some  of  the  input  packets  from  its  input  ports,  performs 
computations,  and  some  time  later  sends  packets  from  its  output  ports.  Then  it 
changes  its  internal  state  and  reenters  the  wait  mode.  In  general,  an  input  port 
can  be  a btiffer  which  can  hold  a number  of  packets  simultaneously.  A packet 
remains  at  an  input  port  until  it  Is  absorbed  by  the  module.  An  output  port,  on 
the  other  hand,  is  more  like  a door  through  which  output  packets  pass. 

The  module  must  make  the  following  declsionst  when  to  fire,  which  input 
packets  to  absorb,  what  computations  to  perform,  the  values  of  the  output 
packets  and  the  times  at  which  they  are  sent,  and  the  new  state  of  the  module. 
These  decisions  can  depend  on  the  following  factorst 

1. )  The  values  of  all  packets  at  the  input  ports. 

2. )  The  time  at  which  each  of  the  input  packets  arrived. 

3. )  The  current  time. 

4. )  The  current  state  of  the  module. 

6.)  Stochastic  processes  within  the  module. 

However,  while  a module  is  in  the  firint  mode,  it  cannot  be  affected  by  input 
packets  which  have  arrived  since  the  module  entered  the  firint  mode. 

These  rules  of  operation  allow  for  modules  whose  behavior  depends  heavily 
on  timei  the  current  time  of  the  module,  and  the  time  at  which  each  input 
packet  arrives.  While  this  does  not  fit  in  well  with  the  philosophy  of 
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tlme-lndependmit  dastgii.  It  will  not  cauM  any  particular  difficulties  for  the 
simulation. 

I 

f A packet  communication  architecture  module  has  only  throe  forms  of  Input 

I 

Informatloni 

' 1.)  The  Initial  state  Sq  of  the  module. 

2. )  The  values  of  the  packets  received  at  each  Input  port. 

3. )  The  time  at  which  each  Input  packet  arrived. 

h 

I Similarly,  It  produces  only  three  types  of  output  information: 

f 

1. )  The  final  state  Sj  of  the  module. 

2. )  The  values  of  the  output  packets  sent  from  each  output  port. 

3. )  The  time  at  which  each  output  packet  Is  sent. 

The  output  information  produced  by  a module  can  depend  only  on  the  Input 
Information  and  the  stochastic  processes  within  the  module.  If  the  module 
contains  no  stochastic  processes,  then  the  simulation  of  the  module  should 

■ 

produce  the  correct  output  Information  based  on  the  Input  Information.  If  the 
module  contains  stochastic  processes,  then  the  simulation  should  produce  the 
correct  output  information  based  on  the  Input  Information  and  one  set  of 
choices  for  the  random  variables.  Furthermore,  the  stochastic  processes  should 
be  simulated  In  such  a way  that  the  values  of  the  random  variables  are  chosen 
with  the  same  probability  In  the  simulation  as  they  would  be  In  the  actual 
module. 
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Module  History 

Tho  Input  and  output  infonnatlon  raoalvad  and  sent  by  a module  while  It 
la  operating  can  be  formally  deacrlbed  In  terms  of  histories.  The  history  of  a 
single  port  Is  a swiuence  of  ordered  palrsi 

h ■ j)  I t • • • I ISyfly)  • • • • • 

where  Xj  is  the  data  value  contained  In  the  7th  data  packet  arriving  at  or  being 
sent  from  the  port,  and  tj  is  the  time  at  which  It  is  received  or  sent.  Since 
packets  are  sent  or  received  one  at  a time,  we  have  tj  > ry.y,  for  all  7 2 1. 
We  also  require  > 0.  This  implies  that  no  output  port  can  produce  a packet 
at  time  0.  This  restriction  is  part  of  the  finite  delay  restriction  which  will  be 
discussed  In  Chapter  3.  Furthermore,  no  Input  port  can  receive  a packet  at  time 
0.  Any  packets  present  at  an  Input  port  Initially  are  considered  part  of  the 
module’s  Initial  state,  and  not  part  of  the  Input  port's  history. 

While  similar  In  Idea,  this  definition  of  history  differs  from  the 
definitions  used  by  Patll  [18]  and  Kahn  [12]  in  their  work  with  determinate 
systems.  Their  histories  are  sequences  of  data  values  only  and  contain  no  time 
values.  Histories  without  time  values  were  useful  for  them,  since  determinate 
systenu  have  time-independent  behavior.  For  simulation  purposes,  however,  the 
simulation  of  the  timing  is  as  Important  as  the  simulation  of  the  data 
operations.  Moreover,  the  time  values  are  part  of  the  Input  and  output 
Information  of  the  module.  Hence,  the  time  values  are  an  Important  part  of  the 
history. 


Since  an  Infinite  number  of  data  packets  could  eventually  pass  through  a 
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port,  a history  can  ba  an  infinite  ssquenoe.  Howevar,  for  any  physical  system, 
thara  must  ba  soma  minimum  saparatlon  tima  8 batwaan  any  two  packats. 
Hanca,  no  mora  than  r/8  packats  can  pass  through  tha  port  bafora  tima  r.  This 
Implias  that  a history  must  ba  a countabla  saquenca. 

The  history  of  an  input  port  is  denoted  hij^,  and  the  history  of  an 
output  port  0^  is  denoted  hoj^.  The  input  history  of  a module  M with  input 

ports  tj,i2 ^ ^ n«tupla  of  tha  histories  of  the  input  portst 

HI  - <hi|,hi2 

Similarly  the  output  history  of  a module  M with  output  ports  ******  ^ ^ 
m-tuplet 

HO  - <hQ|,h02 ho^. 

Just  as  tha  histories  of  the  input  ports  to  a module  can  ba  combined 
together,  tha  histories  of  tha  system  input  ports  (those  input  ports  which 
receive  packets  from  an  external  source  rather  than  from  other  modules  in  tha 
system)  can  be  combined  into  a system  input  history 

I “ hlj^,  • • • , hi j>, 

where  .••,<2  Oia  system  input  ports.  Similarly,  tha  histories  of  the 
system  output  ports  can  be  combined  into  a system  output  history 

0 - <hOf,ho^,  . . . ,h02>, 
where  . . . ,0^  are  the  system  output  ports. 

It  will  be  useful  to  define  the  relation  "is  an  initial  segment  of”  between 
two  histories.  First,  a history  hj  la  a proper  initial  sesment  of  a history  h2t 


denoted  hj  c h2.  if 


hj  • j)  • t • • • # ^ • 


and  either 


or 


♦ • • * • • • • • • • 


h^  ■ tjjJ  t f • • • * f ty^j)  t • • • • 

Then  hy  is  an  initial  segment  of  h^,  denoted  hy  C h^.  if  hy  c h2  or  hy  * h2« 


These  relations  can  be  extended  to  module  inpnt  and  module  output 
histories  as  foUowsi 
If 

HI  ■ <hiyihi2»*>>>hij|> 

Hr  - <hi^,hl^ hi;;> 

then  HI  C HI’  if  and  only  ift 

hiy  C hiy,  for  all  Isfin. 

The  definitions  for  module  output,  system  Input,  and  system  output  histories  are 
similar.  Similarly,  we  can  define  the  relation  c over  module  and  system 
histories. 


A final  notation  Is  to  define  the  history  up  to  some  time  f.  For  a single 
port,  h(t)  is  a history,  h’,  where  h'  contains  all  elements  in  h with  time 
values  s t.  Hence  h(()  C h.  This  idea  can  be  extended  to  module  histories,  as 
wells 

HIU)  - <hiy((),  hi2(t) hij,(f)>  . 


Thus  HI(f)  C HI  - HI(«). 


Uslnf  th*  Botton  of  hlxtoites,  th«  operation  of  a packet  enhimniitr^tion 
architecture  module  can  ka  atated  ^eciaeljr.  If  the  module  enntiiiim  no 
stochastic  processes,  then  the  output  history  HO  and  the  final  state  Sj  are 
functions  of  the  input  history  HI  and  the  initial  state  S^.  For  modules 
containing  stochastic  processes,  HO  and  Sj  are  functions  of  HI,  S^,  and  the 
values  of  the  random  variables. 

Note  that  a module  which  computes  a function  over  histories  as  they  are 
defined  here  may  not  compute  a function  over  the  histories  defined  by  Patil 
[18]  and  Kahn  [12].  Since  our  histories  Include  time  values,  modules  such  as 
arbiters  and  time  clocks  compute  functions  over  these  histories,  whereas  they 
are  not  functional  over  histories  without  time  values. 

Ckannol  Oporaiion 

In  a packet  cemmunication  archltacture  sirstam,  a communication  channel 
serves  only  to  carry  the  output  packets  from  an  output  port  of  one  module  to 
an  input  port  of  another  module.  Furthermore,  the  channel  preserves  the 
ordering  of  packets.  Packets  will  be  received  at  the  input  port  in  the 
order  in  which  they  sent  from  the  output  port.  A channel's  operations  can  be 
stated  formally  in  terms  of  histories.  If  output  port  of  module  is 

connected  to  input  port  of  module  and  has  output  history 

, 1x^,1^)  , • • • , {x jft j)  , • • • , 

then  If,  will  have  an  input  history 

“ ixj, j) , (xj^f ( 

Due  to  the  order  preservation,  fj>  ('y.j. 


Furthermore,  since  values  cannot  be 
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ncalvwl  'iMfors'*  they  ai«  Mat,  t’j  i tj. 

White  a communlcatloa  chaaael  caaaot  chaa^a  the  values  of  data  packets 
or  their  orderlad.  It  caa  latroduce  a delay  betweea  the  time  at  which  they  are 
seat  aad  the  tliae  at  which  they  are  received.  This  delay  must  be  simulated, 
alace  It  will  affect  the  laput  history  of  the  module  to  which  It  Is  coaaected. 
The  commualcatloa  chaaael  caa  be  simulated  by  oae  of  several  meaas.  First, 
we  caa  simply  l^aore  the  delay  aad  coaslder  hi,.  - ho^  This  would  be 
appropriate  la  cases  where  the  delay  time  of  the  chaaael  is  much  smaller  thaa 
the  delay  time  of  the  modules.  For  example.  If  the  modules  are  close  together 
aad  directly  wired  to  oae  aaother,  the  chaaael  delay  time  will  be  vpry  small. 
Secoad,  we  caa  simulate  a aiodule  aad  the  chaaaels  coaaected  to  Its  output 
ports  as  a single  unit.  Conceptually  we  caa  view  this  as  extending  the 
boundaries  of  a module  I to  include  Its  output  channels,  as  shown  la  Figure 
2.1.  The  output  ports  of  this  extended  module  M*  are  wired  dlnctly  to  the 
laput  ports  of  other  modules.  This  solution  Is  appropriate  If  the  channels 
connected  to  a module  operate  Independently  of  other  channels  la  the  system, 
such  as  channels  which  are  Implemented  as  FIFO  buffer  units.  Finally,  the 
most  general  approach  would  be  to  slmulato  the  channels  as  If  they  were  packet 
communication  architecture  modules.  This  approach  would  be  required  If  the 
channels  do  not  operate  Independently  of  one  aaother.  For  example.  If  packets 
are  sent  from  oae  module  to  another  over  a network,  such  as  the  ARPA 
network  [IS],  the  delay  time  could  depend  on  the  total  number  of  packets  being 
sent  over  the  network.  In  this  case  we  would  simulate  the  ABPA  network  as  a 
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For  tho  rmoiiulT  of  tlUa  thesis,  it  will  be  assumed  that  the  system  to  he 
simulated  consists  of  a number  of  modules  which  are  Interconnected  by 
sero^elay  channels.  Some  of  these  "modules",  however,  ml^ht  actually  be 
extended  modules  or  communication  channels  which  are  to  be  slmuleted.  Thus, 
if  output  port  0^  of  one  module  Is  omaected  to  Input  port  of  another  module, 
then  hOp  - hi,.. 

Timm  ladopundont  Simulation  of  a Modulo 

The  Idea  of  a history  leads  quite  naturally  to  a means  of  representing 
In  the  simulation.  The  time  at  which  a packet  Is  sent  from  an  output  port 
bo  considered  part  of  Its  vaiue,  rather  than  an  Implicit  property.  Thus,  the 
value  of  a packet  Is  a pair  (x,i),  where  x Is  Its  data  value,  and  t Is  its  time 
value.  By  ez^dtly  provldlnf  this  time  Information  In  each  packet,  a 
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slmnlatton  proc9S8or  can  sUnulata  tha  oparatlon  of  a modnla  without  any 
roal-tlmo  coaatrainta. 

For  oxampla,  auppow  wa  wlah  to  stinulata  a DIVIDE  module  aa  ahown  in 
Figure  2.2.  If  the  aimulatlon  proceaaor  receivea  the  packeta,  (x.18)  and  iy,28}, 
on  Ita  input  porta,  then  it  will  aimulate  the  flrlnd  of  the  module  at  time  28, 
and,  alnce  the  delay  tima  of  tha  module  ia  5,  produce  output  packeta 
^*feod  *ttd  {x/y,2S).  The  aimulatlon  la  not  required  to  operate  at  a 

particular  apoad,  alnce  the  actual  time  at  which  tha  output  packeta  are  aent 

f during  the  aimulatlon  la  not  Important. 

I 


Ftgarm  2.2  - Example  of  Simulation  Module  Operation. 

With  thia  meana  of  almulatlud  the  ttmtwj,  the  output  of  the  aimulatlon  of 
a module  la  the  entire  output  history  of  the  actual  module.  Thla  be 
deacrlbed  formally  by  deflnlnd  simulsUoa  historios.  For  any  port  In  the 
aimulatlon,  the  aimulatlon  hlatory  la  the  aaqiieiu  e of  packeta  ««£  through 
the  porti 


tiS  • (Xj,  1 j)  , t^}  , • , • , tx jft  9 • • • , 
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when  9 < < t2  < •••  < tj  < ...  . If  the  simulation  comctly  simulates  a 

port,  then  hs  - h,  whmre  h Is  the  history  of  the  corresponding  port  In  the 
actual  system. 

Simulation  histories  can  be  defined  for  modules,  too.  The  input  simulation 
history  of  a module  Is  an  n-tuple 

HSI  - <hsi2,hsi2 hsi„>  , 

and  the  output  simulation  history  Is  an  m-tuple 

HSO  - <hsO|,hso2 hso^. 

The  system  Input  simulation  history  SI  and  the  system  output  simulation 
history  SO  an  defined  In  a similar  fashion.  Furthermon,  the  nlatlons  G and  c 
an  defined  over  simulation  histories  In  the  same  manner  as  they  an  over  actual 
histories. 

The  requirements  for  the  correct  simulation  of  a module  can  be  precisely 
defined  In  terms  of  histories  for  modules  with  non-stochastlc  behavlon 

Suppose  an  actual  module  produces  an  output  history  HO  and 
finishes  In  a final  state  Sj  when  It  Is  started  In  some  Initial  state 
Sff  and  receives  an  Input  history  HI.  Then  the  simulation  of  this 
module  must  produce  a simulation  history  HSO,  such  that  HSO  - HO, 
and  It  must  finish  In  S^,  when  It  Is  started  In  state  S^^,  presented 
with  a simulation  history  HSI  - HI  and  then  notified  that  no  more 
Input  packets  will  be  received. 

The  requirement  that  the  simulation  be  notified  when  the  last  packet  has 
been  received  Is  needed  to  prevent  the  simulation  from  hanging  up,  waiting  for 
packets  which  will  never  arrive.  This  will  be  discussed  later  In  this  chapter. 


Without  any  constraints  on  the  times  at  which  Input  packets  arrive  at  tha 
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input  ports  of  the  modules  In  the  simulstion,  there  is  no  guarantee  that  the 
relative  orderings  of  packets  on  different  input  ports  will  be  preserved.  This 
can  lead  to  a problem  of  premature  firing,  in  which  the  firing  of  a module  at 
soma  time  tjlu  is  simulated  before  all  input  packets  with  time  i tjlrt  have 
arrived.  Fnr  example,  if  an  arbiter  la  the  simulation  receives  a packet  (x,10) 
on  one  input  port,  it  might  simulate  the  firing  at  time  tflrt  - 10,  and  (assuming 
it  has  a delay  time  of  2)  send  the  packet  (x,12)  from  its  output  port.  Suppose 
now,  though,  that  a packet  (^,5)  is  received  on  its  other  input  port.  The 
arbiter  has  fired  prematurely  and  the  simulation  cannot  proceed  properly. 

To  prevent  this  problem  of  premature  firing,  the  firing  of  a module  at 
time  tfiu  must  not  be  simulated  until  the  entire  input  simulation  history 
HSKf^rs)  has  been  received.  The  only  way  the  simulation  can  know  it  has 
received  hslj^((/lr<)  on  input  port  is  if  it  receives  a packet  with  time  value  2 
tfirt  on  that  input  port.  Thus  if  the  simulation  stores  the  time  value  of  the 
most  recently  received  packet  on  each  input  port  denoted  then  the 

firing  of  a module  at  time  tfirt  can  be  simulated  if  tfirt  i (tlastj^). 

The  simulation  of  a module  proceeds  as  foUowsi 

1.)  Determine  whether  the  module  can  fire  at  scnne  time  tfirt  s 
iskte  i>a9e<i  on  ihe  data  and  time  values  of  those  packets  at 

the  input  ports  with  time  values  s tfirt,  the  current  state  of  the 
module  S^,  and  the  outcome  of  simulations  of  any  stochastic 
processes. 

Z.)  If  the  module  can  fire,  then  simulate  the  firing  of  the  module 
as  foUowsi 


a).  Remove  the  proper  input  packets  from  each  input  port. 


Only  packets  with  time  value  i tjlrt  can  be  removed. 

b) .  Calculate  the  output  data  values  and  the  output  times. 
These  calculations  can  depend  only  on  input  packets  with 
time  values  s tflrt.  Furthermore,  all  output  times  must  be 
greater  than  tflu. 

c) .  Send  the  output  packets  from  the  proper  output  ports. 

d) .  Calculate  the  new  state 
3.)  Go  to  step  1. 

Assuming  the  simulation  will  produce  the  proper  output  packets  each  time 
it  simulates  the  firing  of  a module,  the  output  of  the  simulation  will  always  be 
an  Initial  segment  of  the  output  history  of  the  actual  module,  that  is  HSO  C HO. 
However,  due  to  the  requirement  that  tfiu  s Uiastj^),  it  is  possible  for  the 
simulation  of  a module  to  hang  up  by  waiting  for  packets  which  will  never 
arrive.  Suppose,  for  example,  that  an  arbiter  in  the  simulation  receives  a packet 
(x,10)  on  input  port  1 but  has  received  no  packets  with  time  greater  than  5 on 
input  port  2.  Then  - 5 < tflre  - 18,  hence  the  firing  of  the  module 

cannot  be  simulated.  If  no  more  packets  are  ever  received  on  input  port  2,  the 
firing  of  the  module  at  time  18  will  never  be  simulated,  even  though  the 
module  is  enabled.  The  simulation  must  be  notified  somehow,  when  the  last 
packet  has  been  sent  to  each  input  port,  so  that  any  remaining  input  packets 


can  be  processed  correctly.  With  this  notification,  the  output  of  the  simulation 
will  be  the  output  history  of  the  actual  module,  in  other  wmds  HSO  - HO. 


1 
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Conolusion 

By  Including  the  simulatloii  time  In  each  data  packet,  the  operation  of  a 
module  can  be  properly  simulated  without  any  real-time  constraints.  Although 
this  requires  each  simulation  processor  to  compute  time  values  as  well  as  data  ^ 

values.  It  enables  us  to  simulate  a wide  variety  of  packet  communication 
architecture  systems  with  complete  accuracy. 
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Chapter  3 

Slmalatlon  of  a Syetem 

Introduotion 

In  the  previous  chapter,  methods  of  simulating  the  components  of  a packet 
communication  architecture  system  were  discussed.  If,  in  an  attempt  to 
simulate  the  entire  system,  these  module  simulations  were  conn.ected  together, 
the  simulation  would  most  likely  deadlock.  This  deadlock  results  when  the 
modules  in  the  simulation  are  waiting  for  packets  from  each  other,  but  none 
can  be  fired  until  one  of  them  produces  more  output  packets.  Unlike  deadlocks 
which  mi£ht  occur  in  the  actual  system,  which  should  be  simulated,  this  form 
of  deadlock,  called  hanging  up,  prevents  the  simulation  from  fully  simulating 
the  activities  of  the  actual  system. 

For  example,  the  simulation  program  for  the  arbiter  in  Figure  3.1  has 
received  a packet  with  time  3 on  input  port  2,  but  nothing  on  input  port  1. 
Hence  tlastj  - 0 < t/lre  - 3,  and  the  firing  of  the  arbiter  cannot  be  simulated. 
However,  no  packet  will  ever  be  received  on  the  other  input  port  until  the 
adder  module  fires,  but  this  will  not  happen  until  the  arbiter  fires.  The 
simulation  has  hung  up.  The  actual  system  would  not  have  deadlocked  under 
these  circumstances,  though.  The  arbiter  would  have  fired  and  sent  the  packet 
(7)  at  time  5 to  the  adder,  which  would  have  fired  at  time  10,  and  so  on. 
The  simulation  has  ceased  operation  at  an  earlier  time  than  the  actual  system 
would  have.  A proper  simulation  would  reach  the  same  state  that  the  actual 
system  would.  Additional  coordination  between  the  processors  is  needed  to 
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Figure  3.1  - Simulation  which  has  "Hung  Up.” 

In  this  chapter,  a means  of  providing  this  coordination  will  be  presented 
which  preserves  the  principles  of  packet  communication  architecture.  Including: 
autonomy  of  modules,  communication  by  packets,  and  tlme-lndepettdence.  One 
further  feature  of  this  coordination  method  is  that  all  coordination  Information 
Is  sent  alon^  the  same  paths  as  the  data  packets  are.  There  la  no  need  for 
additional  communication  links  between  processors. 

For  each  module  to  be  simulated,  a slmtilation  processor  must  perform  two 
types  of  operations:  module  activity  simulation,  and  coordination.  These 
operations  together  comprise  the  activities  of  a process  called  the  Btmulation 
modulm.  If  the  simulation  Is  Itself  to  be  a packet  communication  architecture 
system,  each  simulation  modtile  must  be  a packet  communication  architecture 
module.  This  means  that  the  simulation  modules  can  be  viewed  as  autonomous 
processes,  even  If  several  of  these  processess  are  executed  by  one  simulation 


procassor. 


Coordination  Algorithm 

The  simalatlon  hangs  up  when  the  simulation  modules  fall  to  conununlcate 
their  status  to  each  other  but  Instead  wait  passively  for  other  simulation 
modules  to  take  action.  Instead,  the  simulation  modules  could  send  status 
Information  to  each  other  In  the  form  of  time  packets  of  the  form  it) , where  t 
is  a time  value.  Time  packets  are  sent  along  the  normal  communication  links 
between  simulation  modules.  When  a simulation  module  sends  a time  packet 
U)  from  an  output  port,  this  Indicates  that  no  packets  with  time  values  less 
than  or  equal  to  t will  be  sent  from  this  output  port  In  the  future. 

At  any  point  In  the  simulation  that  a module  Is  In  the  wait  mode.  If  there 
Is  no  value  of  tjlrs  s tmtn  • Ulastn)  for  which  the  module  can  fire,  then 

the  module  cannot  possibly  fire  before  or  at  time  tmtn.  If  the  module  has  a 
minimum  delay  time  delay  between  firing  and  producing  the  first  output 
packets,  then  the  minimum  output  time  Is  given  by  the  formulat 

tout  - tmln  delay 

- Itlastif)  + delay. 

The  simulation  module  cannot  produce  more  output  data  packets  with  time 
values  less  than  or  equal  to  tout,  hence  time  packets  (tout)  can  be  sent  from  all 
output  ports  which  have  not  already  produced  packets  with  time  values  greater 
than  or  equal  to  tout.  Furthermore,  If  the  firing  of  a module  at  some  time  t/lre 
Is  simulated,  but  no  data  packets  are  sent  from  an  output  port  Oj  then  a time 
packet  lt/lre*detay)  can  be  sent  from  Oj,  since  any  future  data  packets  from  this 
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output  port  will  hav«  time  values  greater  than  (/lr«  dtlaj. 

As  lonA  as  all  time  and  data  packets  are  sent  from  each  output  port  of  a 
simulation  module  with  strictly  Increasing  time  values,  and  the  communication 
links  between  the  simulation  modules  preserve  the  ordering  of  the  packets,  the 
value  of  tlasti^  for  an  input  port  is  still  the  last  time  value  received  on  that 
input  port,  either  as  part  of  a data  packet  or  as  a time  packet.  No  new  packets 
can  be  received  at  an  input  port  with  time  values  less  than  or  equal  to  tlastj^. 
If  the  values  of  delay  are  greater  than  zero  for  all  simulation  modules,  then  as  a 
result  of  these  coordination  activities,  the  simulation  modules  will  send 
increasingly  larger  time  values  to  one  another,  until  one  of  the  simulation 
modules  is  able  to  simulate  the  firing  of  its  module,  thereby  avoiding  deadlocks. 

In  the  example  shown  in  Figure  3.1,  The  simulation  module  for  the  arbiter 
has  received  a data  packet  with  time  value  3 on  input  port  2 and  has  received 
nothing  on  input  port  1.  The  arbiter  cannot  possibly  fire  before  time  tmln  - 
m\nlttast^,tlast2)  " ■ 0.  Hence  it  cannot  produce  any  output  packets 

with  time  value  less  than  or  equal  to  tmln  + delay  - 0-t-2  - 2.  Therefore  it  can 
send  a time  packet  (2)  to  input  port  1 of  the  adder's  simulation  module  which 
In  turn  would  update  tlast^  to  2.  The  adder  cannot  possibly  fire  before  time 
tmln  - Min (2, 10)  and  therefore  cannot  produce  any  output  data  packets  with 
time  values  less  than  or  equal  to  tmln  + delay  - 2-f2  - 4.  Therefore  a time 
packet  (4)  can  be  sent  back  to  the  arbiter's  simulation  module  which  would 
then  sat  tlast^  • 4,  and,  since  tflre  -3s  m\nUlastj,flast2i  - Mln(4,3),  the  firing 
of  the  arbiter  module  would  be  simulated. 
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I Th*  optratlon  of  a simulation  module  can  be  stated  as  followss 

1. )  Each  time  a time  or  data  packet  Is  received  on  Input  port 
update  tlastn. 

2. )  Determine  whether  the  module  can  be  safialy  fired.  That  is, 
whether  the  conditions  are  sufficient  for  the  module  to  fire  at 
some  time  tflrt,  where 

tfirt  i 

a. )  If  the  module  can  be  safely  fired,  then  simulate  the 
operation  of  the  module  on  those  input  packets  with  time 
values  s tflr*  and  produce  the  output  data  packets.  For  each 
output  port  Oj  from  which  data  packets  are  sent,  update  the 
value  of  tlast-mUj,  which  is  the  time  value  of  the  most 
recently  seat  output  packet  from  Oj.  For  each  output  port  Oj 
for  which  tlast-outj  < tfirt  + delay,  send  a time  packet  Itflre  * 
delay)  from  o j and  update  tlastmU  j. 

b. )  If  the  module  cannot  be  safely  fired  then  compute  tout, 
where 

tout  - tmin  + delay, 

and  send  a time  packet  (rout)  from  each  output  port  Oj  for 
which  tout  > tlast-outj.  Then  update  the  value  of  tlast-out  j for 
each  of  these  output  ports.  The  value  of  delay  must  be 
greater  than  zero  but  cannot  be  greater  than  the  minimum 
time  required  for  the  module  to  produce  an  output  packet 
after  firing. 

3. )  Return  to  step  1. 

These  coordination  operations  are  quite  simple,  especially  since  time  packets 

I are  produced  primarily  when  the  simulation  module  Is  otherwise  Inactive.  The 

[ 

I simulation  module  must  store  the  value  of  tlastj^  for  each  input  port,  and 

I tlast-out/^  for  each  output  port.  However,  no  storage  for  time  packets  Is  required, 

I since  they  are  not  needed  once  the  values  of  tlastj^  have  been  updated. 

I 

Furthermore,  the  simulation  requires  some  means  of  determining  when  the 
system  input  ports  have  received  their  final  data  packets.  For  instance,  in  the 
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szample  shown  In  Figure  3.1,  the  firing  of  the  arbiter  at  time  3 would  be 
simulated  and  the  packet  (y.S)  would  be  sent  to  the  adder’s  simulation  module, 
as  shown  in  Figure  3.2. 


Figure  3.2  - Simulation  Requiring  Packets  on  System  Input  Ports. 

The  numbers  alongside  the  input  ports  represent  the  values  of  Hast  tor  the 
ports. 


Suppose  that  no  more  packets  are  received  at  input  port  2 of  the  arbiter  (this  Is 
a system  Input  port.)  Then  the  adder  module  will  be  enabled  to  fire  at  time  t/lrs 
• naK(5,10)  - 10,  but  the  simulation  module  cannot  simulate  this  firing,  since 
Hastg  m S < tflrt  - 10.  Instead,  a time  packet  with  value  ain(5,i0)  + 2 will  be 
sent  to  the  arbiter's  simulation  module.  This  simulation  module  will  compute 
tout  • Min(7,3)  -f  2 - 5,  hence  no  time  packet  will  be  sent.  Once  again,  the 
simulation  has  hung  up.  The  simulation  module  for  the  arbiter  Is  still 
expecting  data  packets  on  Input  port  2,  but  none  will  ever  arrive.  In  order  for 
a simulation  to  complete  all  operations  up  to  some  time  tjtnai  time  packets  with 
value  i tflnal  must  be  sent  to  all  system  input  ports  after  the  last  data  packets 
have  been  sent.  If  we  want  to  simulate  the  entire  operation  of  the  system. 
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time  packets  with  value  oo  must  be  sent  to  all  system  Input  ports,  where  oo  is 
greater  than  any  othar  time  value.  This  can  lead  to  a nonterminating 
simulation  In  which  the  simulation  modules  keep  sending  time  packets  to  one 
another  Indefinitely,  even  though  no  modules  will  ever  be  enabled  to  fire  again. 
A means  of  terminating  the  simulation  will  be  presented  In  Chapter  4. 

In  our  example,  we  want  to  complete  all  operations  with  time  ^10.  If  a 
time  packet  (10)  Is  sent  to  the  arbiter's  simulation  module.  It  would  compute 
tout  - Mint?, 10)  * 2 • B and  send  this  value  to  the  arbiter.  The  adder  still 
cannot  be  fired  safely,  but  a time  packet  with  value  ain(9,10)  ■•■2-11  would 
be  sent  back  to  the  arbiter's  simulation  module  which  In  turn  would  send  back 
a time  packet  with  value  nln(ll,10)  ■•■2-12.  Finally,  tjlre  - 10  s 
m]nitlast^,tlast2)  - eln(12,10),  and  the  firing  of  the  adder  at  time  10  could  be 
simulated. 


With  the  addition  of  time  packets,  the  simulation  histories  contain  more 
than  Just  data  packets.  When  comparing  simulation  histories  to  actual  histories, 
however,  only  the  data  packets  are  of  Interest.  The  function  data  is  applied  to 
simulation  histories  to  give  the  sequence  of  data  packets  (Including  their  time 
values)  contained  In  a slmtUation  history.  For  example.  If 

hs  - (x,1),(3).(7.3O).(z,35),(190), 

then 


data(hs)  - (x,l).(^39),(z,35). 

The  function  data  can  be  applied  to  module  simulation  histories  and  system 
simulation  histories  u well. 
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Features  of  the  Coordination  Algorithm 

This  coordination  ol^orltlini  presorvos  the  phUooophles  of  packet 
communication  architecture  design.  All  coordination  Information  Is  passed 
between  simulation  modules  In  the  form  of  time  pockets.  There  ore  no  time 
constraints  on  the  simulation  modules,  and  the  simulation  modules  con  operate 
Independently.  Furthermore,  the  coordination  operations  for  each  module  ore 
very  simple.  Each  simulation  module  performs  Identical  coordination  operations, 
which  allows  uniformity  In  the  simulation  programs. 

One  further  feature  Is  that  a simulation  module  sends  time  packets  only  to 
those  simulation  modules  to  which  It  also  sends  data  packets,  and  these  time 
packets  ore  sent  over  the  normal  data  paths.  This  not  only  keeps  the  number 
of  Input  and  output  ports  to  a simulation  module  limited.  It  eliminates  the  need 
to  synchronize  the  coordination  Information  with  the  data  Information.  If,  on 
the  other  hand,  time  packets  were  sent  along  some  other  communication  links, 
special  measures  would  be  required  to  prevent  a time  packet  from  arriving  at  on 
Input  port  before  a data  packet  having  on  earlier  time  value  does.  By  sending 
time  packets  along  the  normal  communication  links,  we  use  the  flrst-ln, 
flrst-out  property  of  these  links  to  ensure  the  proper  sequencing  of  time  and 
data  packets. 

Bfflolanoy  of  Coordination 

This  coordination  algorithm  is  rather  Inefficient  In  two  respects.  First,  a 
large  number  of  time  packets  must  be  sent  to  keep  the  simulation  coordinated. 
In  the  example  of  Figures  3.1  end  3.2,  a total  of  seven  time  pockets  were 
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transmlttad  so  that  the  arbiter  and  the  adder  could  each  fire  once.  This  causes 
both  a delay  In  the  simulation  and  a heavy  load  on  the  communication  channels 
between  simulation  modules.  For  larger  simulations,  the  number  of  time 
packets  would  be  overwhelming.  Second,  this  method  does  not  allow  all 
possible  concurrency  In  the  simulation.  For  example,  the  two  modules  shown 
In  Figure  3.3  could  potentially  be  simulated  at  the  same  time.  The  adder  will 
not  fire  until  time  10  and  hence  cannot  produce  a packet  with  time  < 12. 
Therefore,  the  firing  of  the  arbiter  at  time  11  could  be  simulated  at  the  same 
time  as  the  firing  of  the  adder.  With  the  coordination  algorithm  described, 
however,  the  simulation  module  for  the  arbiter  would  receive  a time  packet 
with  value  aln(5,10)  * 2 • 7 and  hence  the  arbiter  would  not  be  simulated 
until  after  the  adder  has  been  simulated.  This  lack  of  concurrency  compromises 
the  efficiency  of  the  simulation,  since  It  causes  the  simulation  processors  to 
wait  unnecessarily. 
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described  only  tivo  properties  ere  assumed  about  the  modules  to  be  simulated! 
they  will  not  produce  any  output  packets  while  In  the  wait  mode,  and  for  each 
module  there  la  some  minimum  delay  time  dtlaj  between  when  It  fires  and 
when  It  produces  the  first  output  packets.  This,  of  course,  makes  the 
coordination  procedures  very  simple,  but  It  creates  the  two  inefficiencies 
mentioned  above.  If,  on  the  other  hand,  we.  make  use  of  the  fact  that  ah  ADD 
module  cannot  fire  without  first  receiving  data  packets  on  both  Input  ports, 
then  for  the  example  In  Figure  3.1,  the  earliest  possible  time  for  It  to  produce 
an  output  packet  could  be  calculated  as 

tout  - maxUlast j.tlast^)  * 2 
- *ax(e,ie)  4-2-12. 

The  time  packet  (12)  could  be  sent  to  the  arbiter's  simulation  module  which 
would  then  fire  the  arbiter  at  time  3 and  send  the  packet  (^,5)  to  the  adder's 
simulation  module.  Furthermore,  an  ADD  module  can  only  absorb  one  data 
packet  at  a time  from  each  input  port,  hence  the  firing  of  the  module  at  time 
10  could  be  simulated  even  though  tlojt^  - 5 < tflro  - 10.  By  making  use  of 
these  two  particular  properties  of  ADD  modules,  only  one  time  packet  would  be 
transmitted  In  the  simulation,  as  opposed  to  the  original  seven. 

Ot  course,  there  is  a trade-off  between  the  complexity  of  the  coordination 
procedures  within  each  simulation  module,  and  the  efficiency  of  the 
coordination.  In  the  most  extreme  case,  each  simulation  module  could  simulate 
the  entire  system  Internally  to  determine  whether  a particular  module  can  be 
safely  fired.  This  would  certainly  minimise  the  amount  of  coordination 
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information  sent  betwaen  simxilation  modules,  but  it  would  be  overwhelmingly 
complex.  In  Chapter  6,  several  refinements  to  the  proposed  coordination  method 
will  be  described.  The  emphasis  will  be  on  refinements  which  do  not  Increase 
the  complexity  much  but  do  Increase  the  efficiency  significantly. 

Correotttess  of  the  Sysitom  Simulation 

The  combination  of  the  module  activity  simulation  and  the  coordination 
operations  for  each  module  will  guarantee  that  when  the  slmulatitm  modules  are 
Interconnected,  they  will  accurately  model  the  activities  of  the  actual  system. 
A proof  of  this  is  presented  in  Appendix  1 and  will  be  described  briefly  here. 

The  proof  applies  only  to  modules  whose  output  history  and  final  state  are 
functions  of  the  input  history  and  initial  state.  The  module  cannot  contain  any 
stochastic  processes.  However,  for  a particular  set  of  choices  of  random 
variables,  the  output  history  and  final  state  of  a module  will  always  be 
functions  of  its  initial  state  and  input  history,  in  which  case  the  proof  will 
•PPly>  If  the  stochastic  processes  are  simulated  in  such  a way  that  the  random 
variables  are  chosen  with  the  same  probability  as  they  would  he  in  the  actual 
system,  the  simulation  will  stochastically  model  the  actual  system. 

To  formally  describe  the  operations  of  the  actual  modules  and  the 
irimnijittowa  of  these  modules,  six  requirements  are  spedfiedi  three  for  the  actual 
modules  and  three  for  the  simulations  of  these  modules. 

For  the  actual  modules,  the  requirements  aret 
1.)  Functionality  of  Outputi  The  output  history  and  final  state  of  a 
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module  depends  only  on  the  initial  state  of  the  module  and  the  input 
history. 

2. )  Monotonicity  of  Output:  The  output  of  a module  at  time  t cannot  be 
affected  by  input  received  after  time  t. 

3. )  Finite  Delay:  The  output  of  a module  at  time  r cannot  be  affected  by 
input  received  at  time  t.  In  other  words,  there  must  be  a finite  delay 
between  the  receipt  of  an  input  pacXet  and  the  production  of  an  output 
packet  which  depends  on  this  input  packet. 

If  a module  satisfies  all  three  of  these  requirements,  then  the  output  history  of 

the  module  up  to  and  including  time  r is  a function  of  the  initial  state  and  the 

input  hlstmry  up  to  but  not  including  time  t. 

These  three  requirements  for  the  modules  to  be  simulated  are  not  very 
restrictive.  The  monotonicity  of  output  requirement  simply  implies  that  a 
module  cannot  look  into  the  future  and  predict  what  input  will  arrive,  nor  can 
it  retract  or  alter  any  output  packets  once  they  have  been  sent  out.  The  finite 
delay  requirement  states  that  a module  cannot  react  instantaneously  to  an  input 
packet.  This  is  true  for  any  physically  realizable  module.  The  functionality  of 
output  requirement  implies  that  the  mod\ile  cannot  receive  any  input 
information  other  than  the  initial  state  and  packets  arriving  at  the  input  ports. 
Furthermore,  the  module  cannot  contain  any  stochastic  processes,  unless  we 
consider  the  operation  of  the  module  for  a particular  choice  of  random  variables. 


For  the  simulation  of  each  module  the  requirements  are:  1 

1.  Correct  Module  Simulation:  The  simulation  of  a module  must  produce 
the  same  data  packets  with  the  same  time  values  as  the  actual  module 
would  for  the  same  input  conditions.  That  is,  suppose  the  simulation  of 
a module  produces  a simulation  history  HSO  when  it  starts  in  initial  state 
Sq  and  receives  an  input  simulation  history  HSI  where  all  of  the  data 
and  time  packets  arriving  at  each  input  port  have  strictly  increasing  time 
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values.  Let 


after  the  Input  simulation  history  HSI  has  been  received.  That  Is,  t/lnal 
Is  the  smallest  of  all  the  final  time  values  received  by  the  Input  ports  of 
the  simulation  module.  Then 

data(HSO(fyin<i/))  . WHtflruU), 

where  HO  Is  the  output  history  of  the  actual  module  when  It  starts  In 
the  same  Initial  state  and  receives  the  Input  history  HI  - data  (HSI). 
Furthermore,  If  tfiiuU  • oo  (all  Input  ports  to  the  module  receive  time 
pacXets  with  value  oo),  then  the  final  state  Sy  of  the  simulation  of  the 
module  will  be  the  same  as  the  final  state  of  the  actual  module. 

2. )  Correct  Orderinf  of  Output  Packets:  If  the  packets  arriving  at  each 
Input  port  of  a module  In  the  simulation  have  strictly  increasing  time 
values,  then  the  output  packets  sent  from  each  output  port  of  the  module 
in  the  simulation  will  have  strictly  Increasing  time  values. 

3. )  Correct  Coordination:  If  a simulation  module  receives  an  Input 
simulation  history  HSI  then  if  tflnd  - ttlaetj^),  eventually  a time  or 
data  packet  with  time  value  greater  than  tflnal  will  be  sent  from  each 
output  port  of  the  simulation  module,  unless  tflnal  - oo,  in  which  case 
time  packets  with  value  oo  will  be  sent  from  all  output  ports  if  the 
corresponding  actual  module  ever  terminates. 


The  first  step  In  the  correctness  proof  Is  to  show  that  the  simulation  and 
coordination  operations  which  have  been  developed  will  fiilfill  the  three 
requirements  for  the  simulation  modules.  Then,  It  Is  proved  that  for  any 
simulation  in  which  the  actual  modules  satisfy  their  three  requirements  and  the 
simulation  modules  satisfy  their  three  requirements,  the  simulation  win 
accurately  model  the  actual  system.  This  is  stated  in  the  following  theorem: 


Theorem  Correctness  of  Simulation. 

Suppose  a simulation  has  the  following  properties: 

1.)  The  modules  to  be  simulated  satisfy  the  monotieity  of  output,  finite 
delay,  and  functionality  of  output  requirements. 


- 51 


2. )  The  simulation  of  each  module  satisfies  the  correct  module  simulation, 
correct  ordering  of  output  packets,  and  correct  coordination  requirements. 

3. )  All  communication  links  between  simulation  module^  operate  properly, 
so  that  if  input  port  1^  la  connected  to  output  port  then  hsl^  - hso^. 

4. )  The  simulation  receives  a system  input  simulation  history  SI,  and 
the  sequence  of  time  values  received  at  each  system  input  port  is  strictly 
increasing. 

(final  - win  Wait 

after  the  system  input  simulation  history  SI  has  been  received,  where 
• • • *1^  sre  the  system  input  ports.  Then  the  simulation  module  for  any 

module  My  in  the  system  will  produce  a module  output  simulation  history  HSOi 
such  that  ^ 

data(HSOy((/lnat))  - HOjitfinal), 

where  HOy  would  be  the  c ^;put  history  of  the  corresponding  module  in  the 
actual  system  under  the  following  conditions: 

1. )  All  modules  in  the  actual  system  are  started  in  the  same  initial  state 
as  the  corresponding  simulation  modules. 

2. )  The  actual  system  receives  the  system  input  history  I where 

I - data(SI). 

Furthermore,  if  tfinal  • oo,  the  final  state  of  each  simulation  module  which 
terminates  will  equal  the  final  state  of  the  corresponding  module  In  the  actual 
system. 

j 

The  theorem  is  proved  by  induction  on  the  sequence  of  ttmw  values 

• • • * 

where  tg  ■ 3,  and 

t0  < tj  < ...  < ti  < ...  i to, 

and  each  time  value  / > 0,  is  contained  in  some  actual  or  simulation  history 
for  the  system.  That  is,  is  contained  in  one  of  the  following  histories:  I, 
the  system  input  history  to  the  actual  system,  HO y,  the  output  history  of  some 
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module  Uj,  SI,  the  system  input  simulation  history,  or  hSOj,  the  output 
simulation  history  of  some  module  My. 


The  Induction  hypothesis  is  as  follows:  For  any  /•  < fif-  such 

that  fy  i tfinal. 


a.)  data(HSOy(/{) ) - HOj(fj),  for  all  modules  My,  ancf 

h.)  Either  - oo,  or  for  any  output  port  o^: 

hso^Cr^)  c hso^. 

In  other  words,  not  only  will  the  simulation  accurately  model  the  actual  system 
up  to  and  including  time  (j,  but  in  addition  the  coordination  operations  will 
cause  each  simulation  module  to  send  packets  with  time  values  greater  than  f| 
from  all  of  its  output  ports.  Thus  the  simulation  cannot  hang  up  due  to  a 
simulation  module  waiting  for  an  input  packet  with  time  value  s r|,  as  long  as 
s tflnal.  Therefore,  by  induction,  the  simulation  will  accurately  model  the 
actual  system  up  through  time  tjlnal. 


Conclusion 

By  incorporating  some  relatively  simple  coordination  operations  in  the 
simulation  modules,  the  simulation  will  accurately  model  the  actual  system, 
while  preserving  the  properties  of  a packet  communication  architecture  system. 
As  a result,  however,  the  simulation  might  fall  to  terminate  even  if  the  actual 
system  terminates,  and  the  simulation  will  be  rather  Inefficient.  -These  two 
difficulties  will  be  dealt  with  in  the  next  two  chapters. 
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Chapter  4 

Termination  of  the  Simulation 

Introduction 

Due  to  the  decentralized  and  time-independent  nature  of  the  simulation  and 
coordination  operations,  there  are  conditions  for  which  the  actual  system  will 
eventually  cease  all  operation,  but  the  simulation  will  continue  indefinitely. 
The  simulation  modules  can  keep  sending  time  packets  with  Increasingly  larger 
time  values  to  each  other  lon^  after  all  module  activity  simulations  have  been 
completed. 

For  example,  in  system  of  Figure  4.1  the  system  input  port  (input  port  2 
of  the  arbiter)  has  received  a time  packet  with  value  oo  and  the  simulation 
module  for  the  switch  has  produced  a data  packet  (x,37).  As  can  clearly  be 
seen,  all  data  operations  by  modules  in  the  system  have  been  completed.  The 
simulation,  however,  will  keep  going.  The  arbiter  will  send  a time  packet 
with  value  Min(108,oo)'fl  - 101  to  the  functional  operator.  This  operator  will 
send  a time  packet  with  value  lOl-fZ  - 103  to  the  switch,  which  will  send  a 
time  packet  with  value  103'fl  > 104  to  the  next  operator.  This  operator,  in 
turn,  will  send  a time  packet  with  value  104-I-3  - 107  to  the  arbiter.  Then  the 
arbiter's  simulation  module  will  start  this  cycle  over  again,  even  though 
nothing  is  really  being  simulated. 

In  this  chapter,  termination  operations  which  can  be  Incorporated  in  the 
simulation  modules  will  be  ;?eveloped.  These  terminations  operations  guarantee 
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The  elides  represent  time  packets;  the  dots  represent  data  packets;  and  the 
numbers  alongside  input  ports  represent  the  values  of  Hast  for  the  Input 
ports. 


that  the  simulation  will  eventually  terminate  If  the  actual  system  does,  while 
preserving  both  the  correctness  of  the  simulation  and  the  principles  of  packet 
communication  architecture.  Furthermore,  as  with  the  coordination,  all  control 
information  is  sent  between  simulation  modules  along  the  normal  data  paths. 
No  special  hardware  Is  required  for  termination,  only  additions  to  the  simulation 
programs.  The  last  part  of  this  chapter  describes  a proof  of  correctness  for  the 
termination  operations.  The  full  proof  la  included  In  Appendix  2. 

If  there  were  some  means  of  simultaneously  observing  all  simulation 
modules  and  all  communication  links  between  them,  then  it  could  be  determined 
when  the  simulation  has  completed  all  data  operations.  The  simulation  has 
completed  all  data  operations  and  can  be  safely  terminated  once  It  reaches  a 
point  where:  all  system  Input  ports  have  received  time  packets  with  value  oo, 
no  modules  have  sufficient  data  packets  to  fire,  and  there  are  no  data  packets  In 


transit  between  the  simulation  modules.  This  omniscient  observer,  however, 
would  not  be  in  keeping  with  the  philosophies  of  packet  communication 
architecture  desl^.  For  our  simulation,  the  simulation  modules  must  send 
control  information  to  each  other  to  determine  whether  the  termination 
conditions  are  satisfied.  Furthermore,  these  termination  operations  must  be 
tlme^lndependent. 


Most  of  the  standard  methods  of  determining  whether  a system  Is  active, 
such  as  time-outs,  or  waiting  for  a maximum  count  on  the  number  of  time 
packets  will  not  work  for  this  simulation.  There  are,  however,  special  features 
of  packet  communication  architecture  modules  which  can  be  taken  advantage  of. 

Connectivity  Classes 

A module  U2  can  only  receive  input  Information  In  the  form  of  packets 
arriving  at  its  Input  ports.  Hence  if  there  is  no  path  from  module  to  M^, 
then  the  activities  of  ll|  cannot  affect  those  of  M^.  To  make  use  of  this  idea, 
the  meaning  of  path  must  be  defined  more  formally.  First,  a module  "is 
eonructtd  to”  a module  M2  denoted  M^  — > N2.  if  an  output  port  of  module  is 
connected  to  an  Input  port  of  M2.  There  is  a path  from  a module  M|  to  a 
module  M2,  denoted  M|  M2,  if  there  exists  a sequence 

M|.M,.M^ M^,M2. 

such  that 

M|  Mj  M^  ...  M,  ->  M2. 

All  communication  with  a module  Is  In  the  form  of  data  packets  travelling 
along  data  channels.  Hence  If  there  is  no  path  from  Mj  to  M2,  then  there  is  no 
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way  for  to  send  infomuitloii  to  H2,  either  directly  or  Indirectly. 

The  difficulties  In  terminating  the  slmulaUon  arise  when  the  system 
contains  cycles.  A module  Is  contained  within  a cycle  If  there  la  a path  from 
one  of  Its  output  ports  to  one  of  its  Input  ports,  that  la  My  My.  For 
example  the  ssrstem  of  Figure  4.1  has  a cycle  01  — > SI  — > 02  A1  -4  Ol. 
The  simulation  modules  contained  In  cycles  ivlU  not  normally  terminate  - they 
cannot  send  time  packets  (00)  untU  time  packets  (oo)  have  been  received  on  all 
Input  ports,  but  the  slmulaUon  modules  wUl  not  receive  these  time  packets 
unless  they  send  them  out.  Instead,  the  simulation  modules  will  keep  sending 
time  packets  with  values  less  than  w around  the  cycles  Indefinitely. 

The  cycles  In  the  system  can  be  identified  by  looking  at  the  e«iulvalence 
classes  formed  by  the  relation  A where  My  M^  If  and  only  if  either  My  - 

M2  (they  are  the  same  module),  or  My  M^  and  M^  My.  This  relation  la 
indeed  an  equivalence  relation  [17],  It  Is  reflexive,  symmetric,  and  transitive. 
Hence  It  defines  a set  of  equivalence  classes  which  are  called  connectivity 

classes  and  are  denoted  Cy.C^ C^.  For  any  connectivity  class  containing 

more  than  one  module,  any  two  modules  In  the  class  must  have  paths  to  each 
other.  That  la,  If  My.  M^  c Cy,  then 

^2  **i- 

An  example  of  a systam  divided  into  Its  connecUvlty  classes  Is  shown  In  Figure 
4.2. 


The  relation  can  be  extended  to  connecUvlty  classes.  Cy  -4  Cy  If  and 
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only  if  My  for  awry  M|cCf,  ^ if  1^  My  for  any  M^cC^, 

MycCy,  then  Cy.  Moreover,  If  Cy,  then  Cy  -f*  C^,  or  else  they 

would  not  be  separate  equivalence  classes.  Thus,  If  Cy,  then  the  modules 

In  C|  are  not  affected  in  any  way  by  the  modules  in  Cy.  We  can  terminate 
the  modules  in  C^  without  worrying  about  the  modules  in  Cy. 

Usin£  the  properties  of  connectivity  classes,  the  conditions  for  terminating 
a connectivity  class  Cy  can  be  stated.  When  all  of  these  conditions  are  satisfied, 
the  simulation  modules  in  the  class  can  safely  terminate. 

la. )  All  system  input  ports  which  are  input  ports  to  modules  in  Cy 
have  received  time  packets  with  value  oo. 

lb. )  All  classes  C^  such  that  C^  Cj  have  been  terminated. 

2. )  No  module  My  t Cy  has  sufficient  data  packets  to  fire. 

3. )  None  of  the  channels  connected  to  input  ports  of  the 

simulation  modules  in  Cy  contain  data  packets. 

If  there  were  some  means  of  detecting  when  a connectivity  class  could 
be  terminated,  then  all  simulation  modules  in  the  class  could  send  out  time 
packets  (oo)  from  all  of  their  output  ports.  In  this  case,  termination  conditions 
la.)  and  lb.)  would  be  identical,  from  a connectivity  class'  point  of  view.  That 
is,  an  input  port  to  a module  My  < Cy  receives  packets  from  one  of  three 
sourcesi  a source  external  to  the  system,  a module  M|  < C^  where  C^  -A  Cy,  or 
a module  M^  c Cy.  In  the  first  case,  is  a system  input  port  and  hence  would 
receive  a time  packet  with  value  oo.  In  the  second  case,  the  input  port 
would  receive  a time  packet  with  value  oo  once  the  connectivity  class  C^  has 
been  terminated.  Conditions  la.)  and  lb.)  can  therefore  be  restated  asi 
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1.)  Time  packets  with  value  » have  been  received  on  all  those 
Input  ports  of  modules  In  the  class  Cj  which  are  not  connected  to 
output  ports  of  other  modules  In  the  class. 

No  special  communication  other  than  time  packets  Is  needed  between 

connectivity  classes  or  with  the  external  world  for  termination.  All  that  la 

needed  to  terminate  the  simulation  of  a system  Is  some  means  of  detecting 

when  the  modules  In  each  class  can  be  terminated. 

If  a class  Cj  contains  only  a single  module  Uj  then  this  module  either  Is 
not  contained  In  any  cycle  In  the  system,  i.e.  My  -h  My,  or  It  Is  part  of  a 
self -loop.  In  which  there  Is  a channel  connecting  an  output  port  of  the  module 
to  an  Input  port  of  the  module,  so  that  My  — > My.  In  the  first  case,  the  normal 
coordination  operations  of  the  simulation  module  are  sufficient  for  termination. 
Since  no  Input  ports  to  the  module  are  connected  to  output  ports  of  modules  in 
the  class,  time  packets  with  value  co  will  eventually  be  received  on  all  Input 
ports  of  the  module.  The  firing  of  the  module  at  any  time  s oo  will  then  be 
simulated.  Then,  since  tout  - oo,  time  packets  (co)  will  be  sent  from  all  output 
ports,  and  the  simulation  processor  can  terminate  the  simulation  of  this  module. 
Thus,  no  special  termination  procedures  are  required  for  modules  which  are  not 
part  of  a cycle  In  the  system. 

For  modules  which  are  part  of  a self-loop  and  for  connectivity  classes 
with  more  than  one  module,  however,  the  normal  coordination  operations  are 
not  sufficient  for  terminating  the  module  simulations.  For  example,  the 
modules  In  Figure  4.1  are  all  In  the  same  connectivity  class  and  therefore 
would  not  terminate.  Those  Input  ports  which  are  connected  to  output  ports  of 


modules  in  the  dess  will  never  receive  time  packets  with  value  oo  without 


special  termination  procedures. 

Termination  Algorithm  for  Conneotlvlty  Classes  Containing 
Cycles 

A means  of  Incorporating  termination  operations  Into  the  simulation  module 
for  each  module  In  a connectivity  class  Cj  will  now  be  given.  This 
termination  algorithm  requires  no  changes  In  the  topology  of  the  system.  There 
Is  no  need  to  add  more  modules  or  communication  links  to  the  system.  Unlike 
the  coordination  operations,  the  termination  operations  are  not  Identical  for  each 
simulation  module.  First,  one  of  the  modules  In  the  class  Is  designated  as  the 
termination  control  module,  denoted  T,  for  the  class.  Any  of  the  modules  In 
the  class  can  be  chosen  for  this  role.  The  simulation  module  for  this  module 
must  initiate  and  validate  the  tests  for  completion  of  all  operations  by  the 
modules  In  the  class.  Next,  for  each  module  In  the  class  other  than  T,  one  of 
the  output  ports  of  the  module  must  be  selected  as  the  sifnal  output  port  of  the 
module.  These  signal  output  ports  must  be  selected  In  such  a way  that  If  we 
look  only  at  the  modules  in  the  class,  there  Is  a path  from  every  module  to  T 
following  only  channels  connected  to  the  signal  output  ports  of  the  modules. 
Finally,  for  each  module  in  the  class,  we  must  determine  which  Input  and 
output  ports  are  connected  to  output  and  Input  ports  of  other  modules  in  the 
class.  The  set  of  all  Input  ports  of  which  receive  packets  from  modules  in 
the  class  Is  denoted  fremjelassj.  Similarly,  the  set  of  output  ports  of  My  which 
sand  packets  to  other  modules  In  the  class  Is  denoted  tojclass  y. 
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The  tenoinatlon  operations  for  the  simulation  module  of  the  termination 
control  module  T are  as  follows: 

1. )  Perform  normal  simulation  and  coordination  activities  until 
every  Input  port  which  Is  not  In  fromjelass;-  has  received  a time 
packet  with  value  oo. 

2. )  When  there  Is  no  way  for  the  module  to  fire  without 
receiving  more  data  packets,  send  test  packets  (te8t.+)  from  all 
output  ports  In  tojclassj-. 

3. )  Walt  until  K test  packets  have  been  received  on  the  Input 

ports,  where 

K - 1 + ^ (itojclass^l  - l). 

ll,cC^ 

In  this  equation,  Itojclass^l,  is  the  number  of  output  ports  of 
module  which  are  connected  to  Input  ports  of  other  modules  In 
the  class. 

4. )  If  any  data  or  time  packets  are  received  while  waiting  for  the 
test  packets,  continue  with  the  simulation  and  coordination 
operations  for  the  module. 

6.)  Determine  the  validity  of  the  test  as  follows: 

a. )  If  all  K test  packets  have  value  teet.-f,  and  no  data 
packets  were  received  while  waiting  for  the  test  packets, 
then  send  time  packets  (oo)  from  all  output  ports  of  the 
module. 

b. )  If  at  least  one  of  the  returning  test  packets  has  value 
test.-  or  a data  packet  was  received  while  waiting  for  the 
test  packets,  then  send  packets  (reset)  from  all  output  ports 
In  to.classj',  wait  for  K (reset)  packets  to  return,  and  go  to 
step  1. 

6.)  Once  time  packets  (oo)  have  been  received  on  all  Input  ports  of 
the  simulation  module,  terminate  the  simulation  of  the  module. 

For  every  other  module  My  In  the  class,  the  termination  operations  for  the 

siBulatlon  module  are  as  follows: 


1.)  Perform  normal  simulation  and  coordination  operations  until  a 


tsst  packet  is  received  on  some  Input  port. 

2. )  When  the  first  test  packet  is  received,  continue  simulating  the 
module  until  all  input  ports  which  are  not  in  from_ctass  j have 
received  time  packets  with  value  oo,  and  the  data  packets  present  at 
the  input  ports  are  not  sufficient  for  the  module  to  fire.  Then,  if 
the  test  packet  has  value  test.4-,  and  no  data  packets  have  been 
received  since  the  test  packet  arrived,  send  (test.-*-)  packets  from 
all  output  ports  in  to_cUssy.  Otherwise  send  (test.-)  packets  from 
all  output  ports  in  tojclass^. 

3. )  If  the  module  receives  any  more  time  or  data  packets,  then 

continue  the  simulation  and  coordination  operations  as  before. 

4. )  Any  time  another  test  packet  arrives,  if  the  packet  has  value 
teat.-f,  and  no  data  packets  have  been  received  since  the  previous 
test  packet  was  sent,  then  send  a (te8t.+)  packet  on  the  signal 
output  port.  Otherwise  send  a (teat.-)  packet  on  the  signal  output 
port. 

5. )  When  the  first  (reset)  packet  is  received  on  an  input  port, 
send  a packet  (reset)  from  each  output  port  In  to  .class  j and 
prepare  for  a new  test.  If  any  further  (reset)  packets  are  received 
before  the  next  test,  send  them  from  the  signal  output  port.  When 
new  test  packets  arrive,  return  to  step  2. 

6. )  When  a time  packet  (oo)  is  received  on  any  input  port  in 
from_class j,  send  packets  (oo)  from  all  output  ports,  unless  this  has 
already  been  done. 

7. )  Once  time  packets  with  value  oo  have  been  received  on  all 

input  ports  to  the  module,  terminate  the  simulation  of  the  module. 


During  the  course  of  a test,  unless  some  simulation  module  can  never  be 
terminated,  a test  packet  will  travel  through  every  communication  link  between 
the  simulation  modules  in  the  class.  Hence,  every  simulation  module  will 
receive  at  least  one  test  packet.  Initially,  T sends  out  |to_classj-|  test  packets. 
On  receipt  of  its  first  test  packet,  a simulation  module  li|  will  send  out 
|te.class{|  test  packets,  thereby  "creating"  Itojclass^l  - 1 new  test  packets. 
Thereafter,  it  will  simply  pass  a test  packet  from  an  input  port  to  an  output 
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port.  Hence,  a total  of  K test  packets  will  be  created.  The  values  of  these  test 
packets  will  be  test.-i-  only  If  no  form  of  data  activity  Is  found  anywhere  In 
the  class.  Because  of  the  way  In  which  the  signal  output  ports  are  chosen,  all 
K test  packets  will  be  funneled  back  to  T which  can  then  check  the  test 
results. 

Features  of  the  Termination  Operations 

This  termination  algorithm  preserves  most  of  the  desirable  properties  of  the 
coordination  algorithm.  In  particular,  the  simulation  modules  still  fulfill  the 
requirements  for  a packet  communication  architecture  system.  Although  one 
module  In  each  class  Is  denoted  as  a termination  control  module.  Its  only 
function  is  to  Initiate  and  collect  Information  about  each  test.  This  module  has 
no  ability  to  monitor  other  modules  or  exercise  any  active  control.  Hence,  the 
simulation  modules  are  still  autonomous.  Furthermore,  all  communication  Is  by 
packets,  and  the  operations  do  not  depend  on  any  timing  restrictions. 

As  with  the  coordination  algorithm,  all  termination  control  information  Is 
sent  over  the  normal  data  channels.  This  avoids  the  problem  of  monitoring  the 
communication  links  between  simulation  modules.  Instead,  the  flrst>in,  flrst-out 
property  of  these  links  ensures  that  no  data  packets  will  be  overlooked  while 
they  are  travelling  between  simulation  modules.  No  special  hardware  Is 
required  for  termination  operations,  only  additions  to  the  simulation  modules. 

One  undesirable  feature  of  these  termination  operations  is  their  dependence 
on  the  overall  structure  of  the  system  to  be  simulated.  Whereas  the  simulation 


and  coordination  operations  of  a module  depend  only  on  the  module  Itself,  the 
termination  operations  depend  on  how  the  module  is  incorporated  In  the  system. 
This  compromises  the  modularity  of  the  design  somewhat.  However,  the 
termination  operations  of  a module  can  be  fully  determined  based  on  a very 
limited  amount  of  knowledge  about  the  system,  namely  how  modules  In  the 
ssrstem  are  Interconnected.  No  details  about  the  operations  of  other  modules  in 
the  system  are  required.  Thus,  while  the  incorporation  of  the  termination 
operations  into  the  simulation  modules  will  decrease  the  modularity  of  design, 
this  decrease  will  be  rather  small. 

Bffioienoy  of  the  Termination  Operations 

The  termination  opertatlons  for  the  modules  In  a connectivity  class  are 
designed  to  be  both  simple  and  efficient.  That  Is,  they  will  not  increase  the 
complexity  of  the  simulation  modules  greatly,  nor  will  the  speed  of  the 
simulation  be  decreased  greatly.  The  efficiency  is  a result  of  several  Important 
features.  First,  the  simulation  and  coordination  operations  need  not  be 
Interrupted  while  the  termination  operations  are  taking  place.  Thus,  If  a test  Is 
Initiated  while  modules  In  the  class  are  still  active,  the  simulation  can  keep 
going,  although  at  a slightly  decreased  speed.  Second,  the  operations  are 
designed  to  keep  the  number  of  tests  Initiated  reasonably  low.  The  first  test 
can  be  Initiated  as  soon  as  the  termination  control  module  has  received  packets 
(oo)  on  all  Input  ports  which  are  not  In  fronijcUss^.  However,  all  K returning 
test  packets  will  not  be  received  until  all  modules  in  the  class  have  received 
peckats  (oo)  on  all  of  their  Input  ports  which  receive  packets  from  outside  the 


class,  and  all  modules  at  some  time  have  ceased  data  operations.  Thus  the 
second  test  cannot  be  Initiated  until  the  first  termination  requirement  for  the 
class  Is  satisfied.  Each  successive  test  cannot  be  Initiated  until  the  previous  one 
has  completed.  This  not  only  simplifies  the  termination  operations,  It  limits  the 
frequency  with  which  tests  can  be  Initiated. 

Correotness  of  the  Termination  Operations 

The  addition  of  the  termination  operations  to  the  slm\ilatlon  modules  will 
not  Interfere  with  the  simulation  of  the  system,  but  they  will  cause  the 
simulation  to  terminate  If  the  actual  system  does.  This  Is  stated  In  the 
following  theorem. 


Theorem  2.  Correctness  of  Termination 

a. )  Suppose  a simulation  is  performed  in  which  the  ipcdules  to  be  simulated 
obey  the  three  requirements:  functionality  of  output,  monotonicity  of  output,  and 
finite  delay,  and  the  simulation  and  coordination  operations  of  each  simulation 
modtile  obey  the  three  requirements:  correct  module  simulation,  correct  ordering 
of  output  packets,  and  correct  coordination,  and  furthermore  the  coordination 
operations  of  a simulation  module  cannot  cause  time  packets  (oo)  to  be  sent  out 
by  the  simulation  module  unless 

laSn  - “• 

Then  the  addition  of  termination  operations  to  the  simulation  modules  as 
described  in  Chcpter  3 will  not  cause  any  of  these  requirements  to  be  violated. 

b. )  If  the  actual  system  ever  reaches  a state  in  which  no  modules  in  the 
ssrstem  will  ever  enter  the  firing  mode  unless  more  packets  are  received  on  the 
s3rstem  input  ports,  then  every  simulation  module  In  the  simulation  of  this 
system  will  eventually  produce  time  packets  with  value  oo  on  all  output  ports. 
If  all  system  input  ports  In  the  simulation  receive  time  packets  with  value  oo. 


The  proof  of  this  theorem  Is  Included  In  Appendix  2 and  will  be  described 
here  briefly.  The  termination  operations  for  different  connectivity  classees  are 
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separate,  hence  we  need  only  prove  that  the  operations  are  correct  for  each 
class.  Moreover,  since  the  termination  operations  are  desired  not  to  Interfere 
with  the  normal  simulation  and  coordination  operations,  the  only  possible 
adverse  effect  of  the  termination  operations  Is  to  terminate  the  simulation  too 
soon.  Thus,  proving  the  first  part  of  the  theorem  Involves  proving  that  the 
simulation  modules  In  a class  will  not  terminate  until  a test  of  the  class 
suceeds,  and  that  a test  will  suceed  only  If  the  termination  conditions  for  the 
class  are  satisfied.  In  other  words.  If  the  termination  control  module  T sends 
out  (test.-*-)  packets,  then  all  K returning  test  packets  will  have  value  test.-*- 
only  If  the  termination  conditions  are  satisfied.  Proving  that  a test  of  a class 
will  not  overlook  some  simulation  module  which  Is  not  yet  ready  to  terminate 
constitutes  the  most  difficult  part  of  the  entire  proof  of  correctness 

To  prove  the  second  part  of  the  theorem.  It  must  first  be  shown  that  a 
test  of  the  class  and  a subsequent  reset  will  eventually  be  completed,  unless  the 
termination  conditions  for  the  class  are  never  satisfied.  In  other  words,  any 
time  the  termination  control  module  send.:  out  test  or  reset  packets,  it  will 
eventually  receive  K test  or  reset  packets,  unless  some  simulation  module 
never  receives  a time  packet  (oo)  on  some  Input  port  which  Is  not  in 
rroin_ciass^,  or  some  actual  module  runs  indefinitely.  Thus,  once  the 
termination  conditions  for  the  class  are  satisfied,  any  previous  test  or  reset 
operations  will  be  completed,  and  a new  test  will  be  Initiated.  Furthermore, 
the  reset  operations  must  cause  all  modules  In  the  class  to  receive  at  least  one 
(raeat)  packat  bafora  tha  naw  tast  packats  ara  raceivad.  Finally,  it  moat  be 


shown  that  a test  will  suceed,  once  the  termination  conditions  are  satisfied. 
Conclusion 

The  relatively  simple  coordination  operations  of  Chapter  3,  which  are 
designed  to  heep  the  simulation  from  deadlocking,  created  a much  more  difficult 
problem  of  terminating  the  simulation.  The  solution  of  this  problem  requires 
both  compromising  the  modularity  of  design  of  the  simulation  modules  to  some 
degree  and  also  adding  termination  operations  which  are  more  complex  than  the 
original  coordination  operations.  This  lack  of  modularity  and  greater  complexity 
makes  the  correctness  of  the  termination  operations  more  difficult  to  prove  than 
the  correctness  of  the  simulation  and  coordination  operations. 

However,  the  termination  operations  do  satisfy  the  design  goals  for  the 
simulation.  The  simulation  remains  a packet  communication  architecture  system 
In  which  all  communication  is  In  the  form  of  packets,  the  simulation  modules 
are  autonomous,  and  the  design  is  time-independent.  Furthermore,  while  the 
termination  operations  are  more  complex  than  the  coordination  operations,  their 
implementation  should  not  be  particularly  difficult,  and  they  are  efficient 
enough  to  have  little  effect  on  the  speed  of  the  simulation. 
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Chapter  6 

Improving  the  Efflolenoy  of  the  Simulation 
Introduotlon 

The  coordination  algorithm  of  Chapter  3 is  rather  primitive  in  that  the 
coordination  operations  of  a simulation  module  make  little  use  of  the  properties 
of  the  actual  module,  other  than  its  minimum  delay  time  delay.  This  leads  to  a 
simulation  which  requires  a great  deal  of  coordination  information  to  be  passed 
between  simulation  modules  and  which  unnecessarily  restricts  the  concurrency 
of  the  simulation. 

Any  modification  to  the  coordination  methods  must  preser\^  their  desirable 
properties.  The  coordination  operations  should  be  simple  enough  to  be  easily 
Incorporated  in  the  simulation  program  for  a module.  The  simulation  should 
still  be  a packet  communication  architecture  system,  hence  there  should  be  no 
centralization  of  control  or  timing  restrictions  on  the  simulation  modules  or  the 
communication  links  between  them.  Finally,  the  design  should  be  modular  - 
the  coordination  operations  for  a module  should  depend  only  on  that  module  and 
not  on  the  structure  of  the  rest  of  the  system. 


In  this  chapter,  two  methods  which  can  Increase  the  efficiency  under  some 
conditions  will  be  presented.  These  two  particular  modifications  were  chosen, 
because  they  are  easy  to  implement  and  apply  to  many  packet  communication 
architecture  systems.  It  will  be  shown  that  with  either  of  these  two 


modifications,  the  Correctness  of  Simulation  Theorem,  described  in  Chapter  3, 
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will  still  apply. 

Modules  whlek  Compute  Monotone  Functions 

Many  of  the  packet  communication  architecture  modules  which  have  been 
designed  to  date  compute  monotone  functions  over  their  histories.  That  is,  if 
the  module  produces  an  output  history  HO^  when  given  the  Input  history  HI^, 
and  an  output  history  HO2  when  started  in  the  same  initial  state  and  presented 
with  an  Input  history  HI2,  where 

HIj  C HI2. 

then 

H0|  C HO2. 

Modules  which  compute  monotone  functions  over  their  histories  are 
characterised  by  the  property  that  the  decision  about  which  input  packets  are 
absorbed  from  each  input  port  and  used  in  a particular  firing  is  independent  of 
the  arrival  times  of  any  input  packets. 

In  particular,  any  determinate  module  computes  a monotone  function, 
where  a determinate  module  [12,18]  is  a module  for  which  the  sequences  of 
output  packets  sent  from  the  output  ports  depend  only  on  the  sequences  of 
Input  packets  arriving  at  the  input  ports,  and  not  on  their  arrival  times.  For 
example,  the  functional  operator  and  switch  modules  of  Chapter  1 are 
determinate  modules. 

One  would  expect  many  packet  communication  architecture  modules  to  be 
determinate,  since  they  embody  the  ultimate  form  of  time-independent  operation. 
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For  example,  all  of  the  data  flow  actors  of  Dennis  [S]  have  determinate 
behavior,  so  by  the  Closure  Theorem  of  Determinate  Ssrstems  of  Patll  [18],  any 
module  which  implements  a data  flow  program  must  be  determinate.  One 
important  module  which  does  not  compute  a monotone  function  over  histories 
and  therefore  Is  not  determinate  is  the  arbiter  module.  The  order  in  which 
packets  are  absorbed  and  subsequently  sent  out  depends  on  the  relative  arrival 
times  of  the  packets  on  each  input  port. 


Other  modules  are  nondetermlnate,  but  do  compute  a monotone  function 
over  histories.  For  example,  a system  clock  module  which,  when  It  receives  a 
packet  of  the  form  (request.time),  sends  out  a packet  containing  the  time  at 
which  the  request  packet  arrived,  computes  a monotone  function  over  histories, 
but  its  output  values  depend  on  the  times  at  which  the  Input  values  were 
received. 

Simalsition  of  Modules  which  Compute  Monotone  Funotlons 

If  a module  computes  a monotone  function,  then  It  can  be  safely  fired  in 
the  simulation  as  soon  as  the  necessary  data  packets  have  arrived  at  the  Input 
ports.  There  Is  no  need  to  make  sure  that  r/lre  s Thus,  the 

simulation  module  can  use  any  of  the  Input  data  packets,  and  not  Just  those 
with  time  values  less  than  or  equal  to  Ulastj^), 

For  example.  If  the  simulation  module  for  an  ADD  module  has  received  a 
packet  (x,10)  on  Input  port  1,  and  a packet  (y,20)  on  Input  port  2,  then  there 
Is  no  need  to  wait  until  a packet  with  time  2 20  has  been  received  on  Input 
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port  1.  Instead,  the  firing  of  the  module  at  time  20  can  he  simulated  right 
away,  since  any  data  packet  received  on  input  port  1 would  not  affect  this 
firing. 

As  long  as  this  revised  firing  rule  does  not  cause  any  of  the  three 
reqtilrements  for  the  simulation  module  to  he  violated!  correct  module  simulation, 
correct  ordering  of  output  packets,  and  correct  coordination,  the  Correctness  of 
Simulation  Theorem  presented  in  Appendix  1 will  still  hold.  To  show  that  this 
modification  will  not  violate  the  correct  module  simulation  requirement,  suppose 
at  some  time  a simulation  module  for  a module  which  computes  a monotone 
function  has  received  an  input  history  HSI',  where  HSI’  C H$I,  the  input 
simulation  history  which  will  ultimately  he  received.  Then  if  all  possible 
firings  of  the  module  on  the  data  packets  are  simulated,  and  an  output 
simulation  history  HSO*  is  produced,  the  effect  of  these  activities  will  he  to 
simulate  the  operation  of  the  actual  module  as  If  It  had  received  an  Input 
history  HI*,  where 

HI*  - data(HSI*). 

We  know  that 

HI*  C HI, 

where  HI  - data  (HSI).  Hence,  since  the  module  computes  a monotone  function, 

HO*  C HO, 

where  HO*  is  the  actual  module's  output  history  in  response  to  HI*,  and  HO  is 
the  actual  module's  response  to  HI,  v/hen  started  in  the  same  initial  state.  In 
simulating  the  actual  module's  operations  on  the  history  HI*,  a simulation 
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history  HSO’  has  been  produced  where 

data(HSO’)  - HO’  C HO. 

The  revised  firing  rules  will  not  cause  the  module  to  fire  prematurely.  Thus, 
the  first  requirement,  correct  module  simulation,  will  not  be  violated. 
Furthermore,  this  modification  will  not  affect  the  rules  for  producing  timp 
packets.  Thus,  the  other  two  requirements  will  still  be  valid:  correct  ordering 

I of  output  packets  and  correct  coordination.  The  Correctness  of  Simulation 

j 

Theorem  still  applies. 

This  modification  will  improve  the  efficiency  of  the  simulation  by 

i 

f increasing  the  concurrency  of  module  simulations.  There  is  no  need  for  a 

I module  which  computes  a monotone  function  to  wait  for  time  or  data  packets 

f 

I when  sufficient  data  packets  are  already  present.  Furthermore,  it  actuaUy 

simplifies  coordination  operations,  since  there  is  no  longer  any  need  to  determine 
whether  a module  can  be  safely  fired. 

Strengthoning  the  Caloalatlon  of  the  Minimum  Output  Time 

In  the  coordination  algorithm  of  Chapter  3,  tout,  the  earliest  possible  time 
at  which  the  simulation  could  next  send  out  a data  packet,  is  dalculated  as 

tout  - itlastff)  + dtlaj, 

where  tlast/^  is  the  time  value  of  the  last  packet  received  on  input  port  In 
other  words,  it  was  assumed  that  the  firing  of  a module  might  be  simulated  as 
soon  as  any  packet  arrives  on  whichever  input  port  1^  currently  has  the  lowest 
value  of  tlttttn.  In  many  cases,  however,  the  module  would  not  be  enabled  to 
fire,  even  if  such  a packet  were  received.  For  example,  if  the  simulation 
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module  for  aa  ADD  module  has  not  received  any  data  packets,  and  tlast^  • 100. 
and  tlast2  • 10.  then  the  firing  of  the  module  for  any  time  less  than  or  equal  to 
100  will  never  be  simulated,  even  If  a packet  with  time  value  11  Is  received  on 
Input  port  2.  The  coordination  operations  are  overly  cautlotu.  They  assume 
only  something  which  Is  true  for  any  module  - If  there  are  not  sufficient 
packets  for  the  module  to  fire,  then  the  module  cannot  fire  before  the  arrival 
of  the  next  packet.  If  the  coordination  operations  could  take  advantage  of  the 
firing  requirements  for  a module,  then  It  could  often  calculate  values  of  tout 
which  are  higher  than  those  obtained  by  the  method  of  Chapter  3. 

Any  change  in  the  method  of  calculating  tout,  will  Inevitably  be  more 
complex  than  the  calculation 

tout  - Ulastn)  + dtlaj. 

Hence,  the  strength  of  the  calculation,  that  Is  the  closeness  to  the  maximum 
possible  value,  must  be  balanced  with  the  simplicity  of  the  calculation.  The 
following  method  of  calculating  tout  represents  a particular  compromise  between 
strength  and  simplicity.  It  Is  very  simple  yet  seems  to  be  reasonably  strong  for 
many  modules. 

Expressing  tke  Firing  Requirements 

First,  a method  of  specifying  under  what  conditions  a module  might  fire  la 
required.  For  any  module,  a boolean-valued  function  F can  be  given  which 
takes  as  arguments  the  values  at  pj,  lifin,  where  pj  i*  the  number  of  packets 
present  at  Input  port  tj.  If 


then  the  module  mljBht  fixe  when  pj  packets  are  present  at  each  input  port  Ij. 
If  the  value  of  the  function  Is  false,  however,  then  regardless  of  the  internal 
state  of  the  module,  the  time,  or  any  stochastic  processes  within  the  module.  If 
each  input  port  tj  contains  exactly  pj  Input  packets  for  all  J,  lifin,  and  the 
module  Is  In  the  wait  mode,  then  the  module  cannot  possibly  enter  the  firinf 
mode.  Thus,  as  long  as  the  value  of  the  function  Is  false,  the  module  cannot 
produce  any  packets  until  more  packets  are  received. 

For  example,  an  ADD  module  has  a function 
*^11110^^1*^2*  ■ *Ai^*'* 

It  cannot  fire  unless  each  each  of  the  Input  ports  contains  at  least  one  packet. 
The  arbiter  has  a function 

It  can  fire  If  there  Is  a packet  on  either  Input  port.  As  a final  example.  If  the 
behavior  of  the  module  Is  totally  unpredictable,  a function 

^^«•*Al.^2 N*  ■ 

can  always  be  used.  This  will  apply  even  for  modules  which  can  sometimes 
fire  without  receiving  any  packets,  since  there  are  no  conditions  for  which  the 
value  of  the  function  is  false,  but  the  module  can  fire. 

An  equation  for  tout  can  be  derived  for  a simulation  module.  If  the 
equation  for  F of  the  corresponding  actual  module  Is  expressed  In  the  following 
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formi 

Pn^  ■ 

A A ••«  A (^n^Cjij)] 

V t(^j2Cj2)  ^ ^^2^C22^  a ...  a (^^2Cj,2)] 

• 

V l(^|2C|^)  A ip^C2q)  A ...  A iC,^)]. 

In  which  each  c^j  is  some  constant  greater  than  or  equal  to  zero.  This  form  of 
the  equation  is  called  the  sum  of  products  form.  Note  that  if  c^j  • 0.  then  the 
predicate  tp^ic^j)  must  have  value  true,  thus  these  factors  can  be  omitted  from 
the  equation.  Equations  with  all  factors  of  the  form  (p2^0)  removed  are  in 
reduced  sum  of  products  form.  In  the  preceding  examples,  the  functions  F^g, 
F,^,  and  F,^  are  expressed  in  reduced  sum  of  products  form. 

Many  functions  cannot  be  expressed  in  this  sum  of  products  form.  In 
fact,  only  those  functions  for  which 

• true 

implies  that  for  any  values,  ^ 0. 

F(^2+il|,^2r*’^2 P»*^s^  ■ 

can  be  expressed  in  this  form.  However,  for  any  function  F we  can  always 
find  a "weaker"  function  F’,  such  that  if 

^fpfp2 V “ 

then 

F’ 

and  an  equation  for  F’  can  be  expressed  in  sum  of  products  form. 


A sum  of  products  equation  for  F can  be  translated  into  an  equation  for 


tout  as  foUowss 


tM  . iiAx[  i)  * ). 

where 

- the  earliest  possible  time  value  of  the  /th  packet  on  input  port 

- the  time  value  of  the  /th  packet  on  /j^,  if  / s p^,  or 

- tlMn,  ^ ^ Pk> 

delay  - the  minimum  delay  time  of  the  module,  and 
c ■ any  niunber  greater  than  zero. 

The  second  term  of  the  equation 

ii'iH  te 

represents  the  calulatlon  of  the  minimum  output  time  baaed  on  the  function  F. 
As  will  be  proved  shortly,  for  any  value  (’  such  that 

<■<'«•  i;r« 

\I  Pi^  ia  the  number  of  packets  on  input  port  with  time  values  less  than  or 
equal  to  f,  then 

■ fa  I 88. 

Hence,  the  module  cannot  possibly  fire  again  before  time  tg,  and  no  data  packets 
with  time  values  less  than  or  equal  to  tp  * delay  can  be  produced  by  the 
simulation  module.  Since  all  packets  in  the  simulation  must  be  sent  from  each 
output  port  In  strictly  Increasing  order,  the  term  c Is  reqvdred  for  tout  to  be 
strictly  less  than  the  time  value  of  the  next  data  packet. 

If  the  calculation  of  tout  were  based  only  on  the  function  F,  It  might  be 
overly  cautious.  It  is  possible  for  the  function  F to  have  value  true  even  when 
the  module  cannot  possibly  fire.  In  this  case,  a calculation  of  the  minimum 
output  time  based  on  the  equation  for  F would  give  a value  which  is  too  low. 
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Even  if  the  function  F has  value  true  at  some  point  in  the  simulation,  if  the 
data  packets  with  time  values  less  than  or  equal  to  are  not 

sufficient  for  the  module  to  fire,  then  no  data  packets  can  be  produced  with 
time  values  less  than  or  equal  to  + dtlay.  Thus,  the  calculation  of 

tout  must  take  the  maximum  of  the  two  predictions  of  the  minimiiTn  output 
time  - that  based  on  the  function  F,  and  that  based  on  the  values  of 

For  example,  for  the  ADD  module  the  equation  is 

tout  - OAxl  m\nitlastptlast2)+delaj  j Rax(f||,r2j)-KiWa7-c  ]. 

For  the  arbiter,  the  equation  is 

tout  - ttAX[  m\n{tl<utj,tlast2)*detaj  j »\n{t j^,t2f) *d€laj-€  1, 

> m\n{tt<ut^,tlast2)  * delay. 

This  equation  degenerates  to  the  original  equation  for  tout.  Finally,  for  the  'I 

function  F,^  the  equation  is 

! 

tout  - HAX(  Ulastn)*delaj  ; 0Hlelay-f.  I | 

t 

- * delay. 

This  equation  also  degenerates  to  the  original  equation  for  tout. 

Correctness  of  the  Caloulation 

this  modified  method  of  calculating  tout  will  not  cause  the  simulation  to 
violate  any  of  the  three  requlrementsi  correct  module  simulation,  correct  ordering 
of  output  packets,  or  correct  coordination.  Hence,  the  Correctness  of  Simulation 
Theorem  given  in  Appendix  2 will  still  apply.  Clearly  the  correct  module 
simulation  requirement  will  still  hold,  since  this  modification  wU  not  affect  the 
data  packets  produced  by  the  module  in  the  simulation. 
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As  for  the  correct  orderini  of  output  packets  requirement,  a time  packet 
will  not  be  sent  out  from  output  port  Oj  with  time  value  less  than  or  equal  to 
tlast-out j,  since  this  Is  checked  for  by  the  simulation  module.  The  only  danger 
Is  that  a time  packet  with  value  tout  might  be  sent  out,  and  later  a data  packet 
with  time  less  than  or  equal  to  tout  is  sent  out.  The  original  proof  shows  this 
cannot  happen  for  tout  - (tlast/f)  * dtlay,  hence  the  problem  can  only  occur 
If 

The  claim,  however.  Is  that  for  any  value  t‘  such  that 

^ ^ " isjsq  ^lasn 

If  Is  the  number  of  packets  on  Input  port  with  time  values  less  than  or 
equal  to  r’,  then 

^ipi,p2 Pn^  - M«. 

Hence  the  module  cannot  fire  again  In  the  simulation  at  any  time,  t’  < t^.  To 
show  this,  look  at  any  t^.  for  which 

tjfcp  ■ eax (1  lu  *^9c 

*c,j  iCjj  /iCjj 

By  our  assumption  about  t’,  and  from  the  equation  for  tg 

and  by  definition  Is  the  earliest  possible  time  value  of  the  Cj^^th  data 

packet  on  input  port  tj^.  Thus,  which  Implies  that  the  predicate 

> false,  for  any  J.  lifiq. 

This  means  that  for  any  J,  the  product  term 

ipjicjj)  (p^C2p  A ...  /\  - false. 

Therefore,  F,  which  is  the  sum  of  these  product  terms  must  have  value  false. 


rv 
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No  firing  of  the  module  before  time 

^0  ■ isjsq 

can  be  simulated,  hence  no  data  packets  can  be  produced  with  time  values  i tg 
* delay  can  be  produced.  If 

tout  • t0  + delay  - (, 

and  c > 0,  the  correct  orderinf  of  output  packets  requirement  will  not  be 
violated. 

Finally,  the  correct  coordination  requirement  will  not  be  violated,  since 
^ late  illatln}  + delay  > Itlastn), 

unless  {tlast^)  - ro.  Thus,  the  Correctness  of  Simulation  Theorem  of 

Appendix  1 will  still  hold  for  this  revised  calculation  of  tout. 

Compatibility  with  the  Termination  Operations 

One  difficulty  caused  by  this  revised  calculation  of  tout  is  that  the 
calculation  might  cause  a simulation  module  to  produce  time  packets  with  value 
00  before  time  packets  with  value  oo  have  arrived  on  all  input  ports,  litis  couiu 
interfere  with  the  termination  operations  for  the  connectivity  class.  If  some 
other  simulation  module  receives  one  of  these  time  packets,  it  will  asstime  that 
the  most  recent  test  succeeded  and  will  send  out  time  packets  (oo)  from  all 
output  ports,  which  might  not  be  valid. 

One  way  to  prevent  this  problem  would  be  to  require  that  no  simulation 
module  send  out  (oo)  packets,  until  all  input  ports  have  received  (oo)  packets. 
Instead,  when  tout  - oo,  it  would  send  out  time  packets  it)  where  t is  some 
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"large"  number.  Thla  seems  rather  awkward,  but  It  will  prevent  the  tout 
calculations  from  Interfering  with  the  termination  operations. 

Features  of  the  Calculation 

This  calculation  of  the  minimum  output  time  uses  information  which  Is 
already  available  to  the  simulation  module,  namely  the  time  values  of  each  data 
packet  at  the  input  ports  and  the  values  of  tlastj^^.  No  attesapt  is  made  to 
predict  the  time  value  of  the  lih  packet  U < I,  except  that  it  is  greater  than 
tlastf^.  This  avoids  passing  more  coordination  Information  between  simulation 
modules,  or  requiring  knowledge  of  the  timing  details  of  the  other  simulation 
modules. 

This  calculation  of  tout  is  reasonably  simple,  in  fact  hardly  more  complex 
than  the  original  calculation.  One  reason  for  this  simplicity  is  that  it  ignores 
much  of  the  Information  which  is  available  to  the  simulation  module.  For 
example,  the  data  values  the  input  packets  are  not  considered,  nor  is  the 
state  or  time  of  the  muuuie.  Under  some  circumstances  this  will  lead  to  a 
weaker  calculation  of  tout  than  might  be  possible.  If  the  conditions  under 
which  a particular  module  can  fire  depend  heavily  on  these  factors,  it  would  be 
worthwhile  to  take  these  factors  into  account  when  calculating  totu. 

This  method  of  calculating  tout  will  increase  the  efficiency  of  the 
simulation  in  two  ways.  First,  it  will  decrease  the  number  of  time  packets 
sent  between  simulation  modules.  Not  only  will  the  difference  between 


successive  time  values  tend  to  be  greater,  the  need  to  send  time  values  around 


81  - 


loops  a number  of  times  Just  to  fire  a module  once  can  be  reduced.  For 
example,  suppose  the  module  of  Figure  5,1  obeys  the  function 

fiPpPz)  - A 

Using  the  original  method  of  calculating  tout,  tout  ■ min (10, 100)  + 2 • 12.  Thus 
a time  packet  (12)  would  bo  sent  to  Vi2,  which  would  send  back  a time  packet 
(13)  and  so  on,  until  after  M2  has  sent  30  time  packets,  it  wotUd  finally 
receive  the  packet  (100)  and  the  firing  at  time  100  could  be  simulated.  If 
instead  we  use  the  calculation 

tout  - HAX[  min (10, 100) +2  j max (10. 100) +2-0. 001  I - 101.999, 
the  time  packet  (101.999)  could  be  sent  to  H^*  which  vrould  send  back 
(102.999),  and  the  firing  of  the  modtile  could  be  simulated.  Thus,  the 

reduction  in  the  number  of  packets  sent  during  the  simulation  can  be  very 
Urge. 


Figure  6.1  - System  which  can  be  Simulated  More  Efficiently  with  Stronger 
tout  Calculations. 
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The  second  improvement  In  the  efficiency  comes  in  the  form  of  increased 
concurrency  of  the  simulation.  In  the  previous  example,  would  not  need  to 
wait  for  time  packets  to  cycle  throu^  the  loop  30  times  before  firing. 
Furthermore,  if  there  were  some  module  connected  to  output  port  02  of 
which  is  waiting  for  a time  packet  with  time  greater  than  or  equal  to  50  from 
ll|.  It  would  receive  this  packet  much  sooner.  By  reducing  the  time  spent 
sending  and  waiting  for  time  packets,  the  simulation  modules  can  spend  a 
proportionately  larger  amount  of  time  simulating  the  data  operations  of  the 
modules.  This  would  Increase  the  concurrency  of  the  module  simulations. 

Conclusion 

These  two  modifications  were  chosen,  because  they  can  be  easily 
Implemented  and  make  use  of  properties  which  are  expected  to  be  common  in 
packet  communication  architecture  systems.  Other  modifications  could  Improve 
the  efficiency  cf  the  slmulatlcw  In  othai  cosoa  wiihoui  mm pmTn  i«i ti g ihe 
dcsiirable  properties  of  the  original  method. 
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Chapter  6 
Conclusion 

Instg^hts  and  Afterthoughts 

As  has  been  demonstrated  here,  it  Is  Indeed  possible  for  the  simtilation  of  a 
packet  communication  architecture  system  to  itself  fulfill  the  design 
philosophies  of  packet  communication  architecture.  The  modularity  and 
time-independence  of  the  simulation  allows  it  to  be  performed  by  virtually  any 
computer  system  which  supports  intercommunicating  processes.  Furthermore, 
the  operations  which  must  be  performed  for  each  module  in  the  system  are 
reasonably  simple  and  therefore  can  be  executed  by  small  processors  such  as 
microprocessors. 

The  methods  which  have  been  developed  here  are  very  general  as  well. 
Few  restrictions  are  placed  on  either  the  characteristics  of  the  modules  in  the 
system  or  on  how  these  modules  are  interconnected.  Moreover,  the  methods  are 
provably  correct,  which  is  an  important  feature  for  any  asynchronous,  parallel 
computation,  due  to  the  numerous  and  often  subtle  difficulties  which  are 
encountered  in  the  design  of  such  systems. 

The  coordination  and  termination  operations  are  simple  enough  to  use  only 
a small  fraction  of  the  simulation  module's  processing  time.  However,  it  is 


difficult  to  estimate  what  fraction  of  the  processing  time  will  be  spent  waiting 
for  the  necessary  time  or  data  packets.  This  will  depend  a great  deal  on  the 
structure  of  the  simulation  facility  and  on  the  system  to  be  simulated.  Thus,  it 
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l5  difficult  to  estimate  the  efficiency  of  the  simulation,  that  is  what  fraction  of 
the  processing  time  will  be  spent  simulating  the  activities  of  the  modules. 
However,  considering  the  low  efficiency  of  a simulation  on  a sequential 
computer  system,  the  efficiency  of  the  parallel  simulation  seems  quite  reasonable 
by  comparison. 

Perhaps  the  fundamental  philosophy  which  is  expressed  in  this  work  is 
that  a certain  amount  of  overhead,  that  is  computation  whose  only  purpose  is 
to  maintain  proper  operation  of  the  system,  is  needed  for  all  but  a limited  class 
of  computer  systems.  This  fact  was  accepted  long  ago  by  designers  of 
traditional  computer  systems.  For  example,  many  of  the  functions  performed  by 
an  operating  system  are  overhead.  Such  operations  as  memory  paging  and 
resoiirce  scheduling  are  incidental  to  the  execution  of  a user's  program. 
Similarly,  the  coordination  and  termination  operations  of  the  simulation  modules 
are  incidental  to  the  simulation  of  the  activities  of  the  actual  system.  In  a 
distributed  computation,  the  increase  in  the  system  load  caused  by  the  overhead 
operations  appears  in  two  forms:  as  added  computations  for  the  components  of 
the  system,  and  as  special  control  information  sent  between  the  components. 

These  overhead  operations  are  acceptable  if  they  are  kept  to  a minimum 
and  are  designed  in  such  a way  that  they  both  preserve  the  design  goals  of  the 
system  and  remain  invisible  to  the  user  of  the  sjrstem.  For  example,  the 
amount  of  overhead  in  the  simulation  is  reasonably  small,  the  principles  of 
packet  communication  architecture  are  preserved,  and  the  overhead  operations 


are  invisible  to  people  performing  simulations. 
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The  design  of  overhead  computations  for  parallel  systems  Is  still  In  a 
rather  primitive  state.  Other  parallel  computer  systems,  such  as  nilac  IV  [3], 
are  structured  in  such  a way  that  the  amount  of  overhead  operartions  is 
minimized.  These  systems  contain  central  controllers  which  tightly  control  the 
operations  of  the  components,  thereby  avoiding  the  need  for  the  processors  to 
communicate  their  status  with  one  another.  Because  of  the  rigid  control 
structure,  however,  it  is  difficult  for  the  user  to  program  such  a system  to  run 
efficiently.  These  systems  are  suitable  only  for  applications  In  which  the 
structure  of  the  algorithm  closely  matches  the  structure  of  the  system. 

Packet  communication  architecture  systems,  with  their  decentralized  control 
and  time-independent  operation  are  potentially  much  more  flexible  and  general 
purpose  than  other  parallel  systems.  However,  along  with  this  increased 
capability  comes  a need  for  the  components  of  the  system  to  keep  their 
activities  couidlualed  properly.  The  design  of  overhead  op**»-atioiis  for  these 
systems  requires  an  approach  which  is  totally  different  from  those  used  in 
designing  traditional  systems.  The  overhead  computations  incorporated  In  each 
component  of  the  system  can  utilize  only  a limited  amount  of  Information  about 
the  rest  of  the  system.  For  example,  the  only  Information  about  the  stat\is  of 
the  rest  of  the  system  available  to  the  coordination  and  termination  operations 
of  each  simulation  module  Is  In  the  form  of  time  and  test  packets  received  at 
the  Input  ports.  Overhead  operations  which  can  be  "modularized"  In  this 
fashion  seem  rather  foreign,  partly  because  they  have  no  locus  of  control. 
Instead,  the  operations  take  place  in  many  locations  simultaneously. 
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Furthermore,  while  one  component  of  the  system  Is  performing  operations,  the 
state  of  the  rest  of  the  system  can  be  changing.  The  overhead  operations  must 
be  designed  to  operate  correctly,  despite  a continuously  changing  system  state. 
As  a result,  one  cannot  fully  understand  how  the  operations  work  by  focusing 
on  one  component  at  a time.  The  system  must  be  viewed  as  a whole  to  see 
how  the  operations  work.  For  example,  the  termination  operations  performed 
by  each  simulation  module  make  little  sense  when  viewed  individually,  but 
they  fit  together  Into  a computation  which  will  detect  when  the  simulation  can 
be  terminated. 

To  date,  no  general  techniques  for  designing  the  overhead  operations  In 
packet  communication  architecture  systems  have  been  developed.  Instead,  they 
have  been  designed  on  a case-by-case  basis,  taking  advantages  of  special 
properties  of  the  system.  For  example,  the  design  here  takes  advantage  of  the 
fact  that  the  sole  purpose  of  a simulation  is  to  model  the  behavior  of  some 
other  system.  If  the  actual  system  contains  deadlocks  or  other  malfunctions,  the 
simulation  should  model  these  deadlocks  and  malfunctions.  The  burden  of 
designing  a system  free  of  errors  is  left  up  to  the  system  designer.  In  the 
future,  however,  general  techniques  should  evolve  which  make  the  overhead 
operations  both  easier  to  design  and  understand. 

Suggestions  for  Further  Research 

There  are  two  directions  In  which  further  research  can  build  upon  the 
work  which  has  been  presented  here.  First,  more  work  Is  required  before 
packet  communication  architecture  systems  can  be  simulated.  In  particular,  a 


means  of  programming  the  simulation  modiiles  is  needed.  Ideally,  the  user  of  a 
simulation  facility  should  be  able  to  specify  the  operations  of  the  components  of 
the  actual  system  In  a high-level  language,  such  as  the  Architecture  Description 
Language  of  Leung,  et  ai  [14].  These  specifications  would  then  be  translated 
Into  programs  for  the  simulation  modules  by  an  ADL  compiler.  The  user  should 
not  be  concerned  with  the  coordination  and  termination  operations,  nor  with  the 
details  of  the  module  activity  simulation.  Fortunately,  the  coordination  and 
termination  operations  are  simple  and  rmlform  enough  that  they  will  not 
Increase  the  complexity  of  this  translation  greatly.  The  major  difficulty  Is  the 
design  of  a language  which  allows  the  specification  of  a wide  variety  of 
systems  In  a concise  and  understandable  form,  but  can  be  translated  Into 
programs  for  the  simulation  modules.  With  the  Increasing  Interest  In  parallel, 
asynchronous  computing  systems,  a convenient  and  efficient  means  of  simulating 
them  will  be  reqtiired  to  determine  the  best  designs. 

The  other  potential  direction  for  further  research  Is  to  apply  some  of  the 
techniques  and  Insights  which  have  been  developed  here  to  other  areas.  One 
direct  application  would  be  to  the  simulation  of  systems  which  are  not  strictly 
packet  communication  architecture  systems.  Some  systems  which  are  commonly 
simulated,  such  as  air  traffic  control  models,  have  the  essential  properties  of 
packet  communication  architecture  design.  That  Is,  the  system,  can  be 
subdivided  Into  a number  of  components  which  operate  independently  and 
communicate  with  each  other  only  In  a limited  and  well-defined  manner.  For 
example,  an  air  traffic  control  model  can  be  subdivided  Into  geographic  regions. 
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The  activities  within  each  region  occur  simultaneously  and  Independently.  The 
only  communication  Is  between  neighboring  regions,  and  the  only  way  they 
communicate  Is  by  changing  the  boundary  conditions.  The  simulation 
techniques  which  have  been  developed  here  can  be  applied  directly  to  such 
systems.  This  will  lead  to  a highly  parallel  simulation  which  can  be  executed 
by  a relatively  simple  network  of  computers.  For  the  air  traffic  control  model, 
one  can  envision  a "grid"  of  processors.  In  which  each  processor  .simulates  the 
activities  within  one  geographic  region.  The  simulation  of  an  air  traffic  control 
model  on  a network  of  processors  has  been  studied  In  some  detail  by  Thomas 
and  Henderson  [22].  In  their  system,  different  geographical  regions  of  a 
hypothetical  airspace  are  slmtilated  on  different  Arpanet  processors.  The 
simulator  for  one  region  sends  a message  to  the  simulator  for  an  adjacent  region 
when  a plane  crosses  from  the  first  region  Into  the  second.  To  maintain  proper 
ttmo  .<(ynphmni»it{nn.  one  of  the  simulators  maintains  a global  time  clock  and 
broadcasts  the  simulation  time  to  the  other  simulators  at  regular  intervals.  In 
their  description  of  the  system,  the  authors  note  that  a distributed  approach  to 
time  synchronization  would  be  preferable,  since  this  centralized  approach  tightly 
binds  the  simulators  to  the  global  clock.  It  seems  that  coordination  operations 
along  the  lines  of  those  presented  In  Chapter  3 could  provide  the  necessary 
s3mchronl2atlon.  Each  simulator  would  send  a time  packet  to  the  simulator  for 
each  adjacent  region  Indicating  the  earliest  possible  simulation  time  at  which  a 
plane  could  possibly  cross  from  the  first  region  Into  the  next.  In  this  way,  the 
simulation  can  proceed  without  any  centralized  control  or  real-time  constraints 


on  the  simulators. 


Moving  bejrond  the  field  of  simulation,  there  are  other  areas  to  which 
these  technlipies  and  insists  can  be  applied.  The  problems  of  deadlock,  and 
nontermination  which  were  dealt  with  here  occur  frequently  in  parallel, 
asynchronous  systems.  The  concept  of  adding  overhead  operations  to  a system  to 
prevent  these  problems  can  be  applied  to  other  systems.  For  example,  the 
author  [4]  has  Identified  a deadlock  which  can  occur  when  the  data  flow 
language  of  Weng  [23]  is  extended  to  include  both  cycles  and  nondeterminacy. 
This  deadlock  occurs  after  all  computation  by  the  program  is  completed,  but  the 
program  fails  to  recognize  that  it  is  able  to  terminate.  This  deadlock  can  be 
avoided  by  adding  more  data  flow  actors  to  the  program  to  perform  the 
necessary  overhead  operations  and  terminate  the  program.  In  fact,  these 
overhead  computations  are  very  similar  to  the  termination  operations  of  the 
simulation  modules. 

To  design  the  overhead  operations  for  a wider  class  of  parallel, 
asynchronous  systems,  however,  more  general  techniques  will  be  required. 
Ideally,  a programmer  should  be  able  to  specify  a program  in  a high-level 
language  which  will  then  be  compiled  into  a number  of  separate  module 
programs  which  include  all  of  the  needed  overhead  operations.  These  programs 
could  then  be  loaded  into  the  modules  of  a packet  communication  architecture 
sirstem,  and  the  system  would  then  execute  the  program  in  a highly  parallel 
fashion.  Translating  high-level  languages  which  include  such  features  as  data 
structures  and  recursive  procedure  calls  into  individual  module  programs  will 


pose  many  difficulties. 
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Thus,  while  the  focus  of  this  work  was  on  simulating  a particular  type  of 
computer  system  in  a particular  manner,  some  of  the  techniques  and  concepts 
which  were  developed  here  have  much  broader  areas  of  application. 
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Appendix  1 

Correctness  of  the  System  Simulation 

The  following  proof  shows  that  the  simulation  operations  of  Chapter  Z, 
combined  with  the  coordination  operations  of  Chapter  3 will  give  a simulation 
which  accurately  models  the  actual  system. 

Before  proceeding  with  the  proof,  some  additional  notation  is  needed.  For 
an  input  port  of  a simulation  module,  the  value  of  tlastn  is  the  last  time 
value  received  on  that  input  port.  Thus,  for  an  input  port  simulation  history, 
we  can  define  a function  TIast  where  TlasKhsij^)  equals  the  minimum  value 
of  r,  Bsfsoo.  such  that  hsij^iO  - Similarly,  for  an  output  port  of  a 

module,  tlast-outj.  equals  the  last  time  value  sent  from  the  port.  Thus,  a 
function  T last-out  can  be  defined  for  output  port  simulation  histories,  where 
TIast-out (hso^)  equals  the  minimum  value  of  t,  Bsdco,  such  that  hso^(r)  ■ 

Finally,  for  a module  input  simulation  history  HSI  the  function  Tfinal  Is 
defined  as: 

Tfinal  (HSU  - (Uast  (hsinl), 

where 

HSI  - <hsi|,hsi2.  . • . ,hsl^>. 

This  function  can  be  applied  to  system  input  simulation  histories  as  well. 
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B«qtilr«m«Bt«  of  th«  Biaralatimi 

Th«  corwcfMi  wnet  will  apply  to  stmiilatloiu  which  fulfill  tho  following 
fix  condltlona.  First,  thara  ara  thraa  ooudltloiis  ou  tha  modulas  to  ba  slmulatedt 

1. )  FunctionaHty  of  Outputi  Tho  output  history  and  final  state  of  a 
modulo  depend  only  on  tha  initial  stata  of  tho  module  and  tha  input 
history. 

2. )  Monotonicity  of  Outputi  The  output  of  a module  at  time  t cannot  ba 
affected  by  input  received  after  time  t. 

3. )  Finite  Delayi  The  output  of  a module  at  time  r cannot  be  affected  by 
input  received  at  time  t.  In  other  words,  there  must  be  a finite  delay 
between  the  receipt  of  an  input  packet  and  the  production  of  an  output 
packet  which  depends  on  this  input  packet. 

If  a module  satisfies  all  three  of  these  requirements,  then  its  output  history  up 

to  and  including  time  t must  be  a function  of  its  state  and  its  input 

history  up  to  but  not  Includlnc  time  (.  This  can  be  specified  more  formally  in 

terms  of  histories.  Suppose  for  two  operations  of  a module,  the  module 

produces  an  output  history  HO  when  it  starts  in  initial  state  and  receives  the 

input  history  HI,  and  it  produces  an  output  history  HO’  when  started  in  the 

same  initial  state  Sq  and  given  the  input  history  HI*.  Then  for  any  value  of  t 

such  that 

HlU-«)  - HI*(f-8),  for  aU  s>e, 

the  two  output  histories  must  be  identical  through  time  t,  that  is 

HOU)  • H0’((). 

The  following  conditions  will  be  required  for  each  simulation  module  in 
the  system! 


1.)  Correct  Module  Siwelstiom  The  simulation  of  a module  must  produce 
the  same  values  as  the  actual  module  would  under  the  same 


cirqumstances.  That  la,  suppose  the  stmulation  of  a module  produces  a 
simulation  history  HSO  when  It  starts  In  Initial  state  and  receives 
input  simulation  history  HSI,  where  all  of  the  data  and  time  packets 
arriving  at  each  Input  port  have  strictly  increasing  time  values.  Let 

tflnal  • Tf  Inal  (HSI), 

That  is,  tfinal  is  the  smallest  of  all  the  final  time  values  received  by  the 
input  ports  of  the  simulation  module.  Then 


data(HSO(ry)n(i/))  - H0(ry)na/), 


where  HO  is  the  output  history  of  the  actual  module  when  it  starts  in 
the  same  initial  state  and  receives  the  input  history  HI  - data  (HSI). 
Furthermore,  if  tfinal  - oo  (all  input  ports  to  the  module  receive  time 
packets  with  value  oo),  then  the  final  state  of  the  simulation  of  the 
module  Sj  will  be  the  same  as  the  final  state  of  the  actual  module. 


2.)  Correct  Ordering  of  Output  Packetst  If  the  packets  arriving  at  each 
input  port  of  a module  in  the  simulation  have  strictly  increasing  time 
values,  then  the  output  packets  sent  from  each  output  port  of  the  module 
in  the  simulation  will  have  strictly  Increasing  time  values. 


3.)  Correct  Coordination:  Each  output  port  of  a module  in  the  simulation 
will  eventually  produce  a time  or  data  packet  with  time  value  greater 
than  the  minimum  time  value  of  the  final  packets  received  at  the  input 
ports,  dr  else  the  output  port  will  produce  a time  packet  (oo).  In  other 
words,  suppose  a module  in  the  simulation  receives  an  input  simulation 
history  HSI  and  produces  an  output  simulation  history  HSOw  Then  for 
any  output  port  of  the  module  either 

Tlaet-outlhso^)  > Tfinal  (HSI), 
or 

Tla8t-out(hso^)  - 00. 


The  simulation  and  coordination  operations  (without  the  termination 


operations)  presented  in  (Hiapters  2 and  3,  satisfy  all  six  of  these  requirements. 


as  long  as  the  modules  to  be  simulated  satisfy  the  first  three  requirements. 


First,  the  simulation  operations  developed  in  (Hiapter  2 will  guarantee  that  the 


correct  module  simulation  requirement  Is  satisfied.  To  see  this,  suppose  at  some 


point  in  the  simulation,  a simulation  module  has  received  a simulation  history 


HSI’  where  HSI’  C HSI  (the  ultimate  simulation  history  which  will  be 


received  by  the  simulation  module.)  Assuming  packets  arrive  at  each  input  port 


1 


c 

I 


with  Strictly  lacTMtlBC  tlm  viIum,  then  If 

MM  - Tflnal(HSI’)  - 

no  new  packets  with  time  less  then  or  equal  to  MiM  will  be  received  on  any 

[ 

> Input  port.  By  the  firing  rules  for  the  simulation,  the  firing  of  the  module  at 

time  tflre  cannot  be  simulated,  unless  tflre  i tmln.  Thus,  when  the  firing  of  the 

f : 

I module  at  tlhie  tflrt  Is  simulated  the  simulation  history  HSI  Itflri)  has  been 

F 

received.  Assuming  the  simulation  correctly  simulates  the  firing  of  the  module, 
the  proper  output  packets  will  be  produced.  Furthermore,  once  the  simulation 
module  has  received  the  entire  Input  simulation  history  HSI  with 

tfinal  > Tfinal (HSI), 

the  firing  of  the  module  for  all  values  of  tjlrt  s tfinal  will  be  simulated. 

Hence,  all  output  pockets  with  time  values  less  than  or  equal  to  tfinal  will  be 
produced  In  response  to  this  Input  simulation  history,  thereby  guaranteeing  that 

data(HSO((^iia/))  - H0((^tm/).  i) 

Thus  the  simulation  will  satisfy  the  correct  module  simulation  requirement. 

The  second  requirement,  correct  ordering  of  output  packets,  la  met  as  long 
I as  the  Input  packets  to  the  simulation  module  are  correctly  ordered.  That  is.  If 

1 

an  output  port  Oj  at  the  simulation  module  first  produces  a packet  and  then 
a packet  P2  then  t|,  the  time  value  In  p^,  must  be  less  than  t^  the  time  value 

In  P2.  To  show  this,  four  cases  must  be  conslderedi  P 

i' 

1.  Pj  and  P2  are  both  time  packets. 

Then  P2  would  be  sent  out  only  If  <2  ^ tlast-outj  - f|. 

2.  P|  is  a data  packet  and  P2  Is  a time  packet. 

As  la  case  1,  P2  would  be  sent  only  If  (2  > tlast-autj  • f|. 


3.  P|  and  P2  are  both  data  packets. 

Assomlnd  the  simulation  module  satisfies  the  correct  module  simulation 
requirement,  data  packets  will  always  be  produced  in  tim  proper  order. 


4.  p^  is  a time  packet  and  P2  is  a data  packet. 

p^  was  produced  with  a time  value  - tmin  * ddof  only  if  the  module 
could  not  possibly  fire  before  or  at  time  tmtn.  The  actual  module  always 
has  a delay  time  d>sater  than  or  equal  to  dtlaj  between  firing  and 
producing  output  packets,  hence  the  simulation  module,  could  not  send  out 
a data  packet  P2  with  time  s from  the  output  port  after  Pj  has 
been  sent. 


For  each  of  these  four  cases,  the  simulation  will  satisfy  the  correct  ordering  of 


output  packeU  requirements. 

The  coordination  operations  also  satisfy  the  correct  coordination 
requirement.  If  the  simulation  module  receives  an  input  simulation  history  HSI 


with 


tfiruU  - Tf  Inal  (HSI), 

Uten  after  all  output  data  packets  have  been  produced,  it  will  send  out  time 
packets  with  value 


tout  • tflnal  * dtlttj, 

from  all  output  ports  for  which  tout  > tlost-outj.  Since  <May  is  draater 
zero,  either  tout  > tflnal,  or  tout  • tflnal  ■ 00.  Hence,  after  the  last  time  and  data 
pockets  have  been  sent  from  each  output  port  Oj,  either 

tlast-outj  t tout  > tflnal. 


or 


ttast-outj  • tout  - tflnal  ■ 00. 

Th  r to  correct  coordination  requirement  will  bo  satisfied. 


s 


I 


A jtfoof  can  now  be  given  which  shovrs  that  If  the  modules  to  bo 


slmiiUit»d  satisfy  thalr  thxat  novlnmaats,  aad  tha  slmnlatloaa  of  thaaa  modulas 
satisfy  thalr  thraa  raqulianaats,  than  whan  thaaa  simulation  modulas  ara 
Intarconnactad,  tha  simulation  will  accuratdy  modal  tha  antira  system. 


Theorem  X*  Corractnass  of  Simulation. 

Suppose  a simulation  has  the  foUowiiid  piopertlesi 

1. )  The  modulas  to  be  simulated  satisfy  the  monotieity  of  output,  finite 
delay,  and  functionality  of  output  requirements. 

2. )  The  simulation  of  each  module  satisfies  the  correct  module  simulation, 
correct  orderinf  of  output  packets,  and  correct  coordination  requirements. 

3. )  All  communication  links  between  simulation  modules  <varata  properly. 
In  other  words.  If  input  port  fj^  Is  connected  to  output  port  then  hsi^ 
• hso^. 

4. )  The  simulation  recelvea  a system  input  simulation  history  SI  and  the 
sequence  of  time  values  received  at  each  system  Input  port  Is  strictly 
incraasind. 

Let  tJlTial  m TfinaHSI),  that  is  tfinal  equals  tha  smaUast  final  time  value 
received  by  any  of  the  system  Input  ports  during  tha  simulation.  Then  the 
simulation  module  for  any  modulo  will  produce  a module  output  simulation 
history  HSOj  such  that 

data(HSOy((^na/))  ■ MjUflnal), 

where  HOy  would  be  the  output  history  of  the  corresponding  module  In  the 
actual  system  under  fidlowlng  condltlonst 

1. )  All  modules  in  the  actual  system  ate  started  In  the  same  initial  state 
as  tha  corresponding  simulation  modules. 

2. )  The  actual  system  receives  tha  system  input  history  I,  where 

1 - data(SI). 

Furthermore,  if  tfituU  ■ oo,  tha  final  state  of  each  simulation  module  which 
terminates  will  equal  the  final  state  of  the  corresponding  module  in  the  actual 
systam. 


Proof  of  Lemma  i.i 


The  proof  will  follow  by  induction  on  the  sequence  of  pockets  which  an 
observer  would  see  if  he  were  to  simultaneously  observe  the  output  ports  of 
every  simulation  module.  This  sequence  would  be  of  the  form 

where  Py  is  the  Jth  packet  observed.  In  any  physical  systam, 
no  two  packets  could  appear  at  the  exact  same  time,  so  the  packets  wlU  be 
totally  ordered  in  time.  The  sequence  of  peckets  sent  from  each  output  port  is 
countable,  and  there  are  a finite  number  of  output  ports  in  the  system,  hence 
the  sequence  Py,P2f>  must  be  countable.  This  allows  ns  to  perform  induction 
on  the  sequence. 

Basist  Initially,  no  output  ports  have  produced  any  packets,  thus  no  ordering 
constraints  have  been  violated. 

Indnctlont  Assume  the  observer  has  seen  the  sequence  P|.P2 P|  onA  up  to 

this  point,  all  output  ports  have  produced  packets  with  strictly  Increasing  time 
valuas.  Than,  by  the  flrst«in,  first-out  property  of  the  communication  links,  all 


Input  ports  connscted  to  tlnso  output  ports  hovo  rooaivod  packots  with  strictly 
Increasing  tlmo  values.  Furth«rmore,  all  system  Input  ports  have  received 
packets  with  strictly  Increasing  time  values.  Hence,  whichever  module  produces 
packet  must  have  received  input  packets  at  each  Input  port  with  strictly 
Increasing  time  values  up  to  this  point.  Since  this  simulation  module  satisfies 
the  correct  orderinf  of  output  packots  requirement,  the  time  value  of  P{^|  must 
be  greater  than  the  time  values  of  all  packets  which  have  been  sent  from  this 
output  port  previously. 

Thus,  by  induction,  no  packet  In  tlm  sequence  P|tP2f*  can  violate  the 
ordering  requirements  for  each  output  pnt. 


Lemma  1.2.  MonoUmiclty  of  Simulation  Output. 

If  a module  satisfies  the  monoticity  of  output,  rinite  daisy,  and  functionslity  of 
output  requirements,  and  the.  corresponding  simulation  module  satisfies  the 
correct  module  shnulatien  requirement,  then  the  output  data  packets  produced  by 
a module  in  the  simulation  with  time  values  less  than  or  equal  to  t will  depend 
only  on  the  Initial  state  and  the  input  data  packets  received  with  time  less  than 
t.  More  preclsriy,  suppose 


data(HSI(r-6))  - HI(r-<),  for  all  8>e, 
and 


t i Tfinal(HSI). 

Then,  if  the  actual  module  and  the  slmalation  module  both  start  in  the  same 
Initial  state 

datalHSOlO)  - H0(f), 

where  HSO  is  the  output  simulation  history  of  the  simulation  module  after 
receiving  HSI,  and  HO  Is  the  output  simulation  history  of  the  actual  module 
after  receiving  HI. 


The  Idee  behind  this  lemma  is  that  the  simulation  can  and  will  produce 
the  output  simulation  history  HSOltl,  once  the  input  simulation  history  HSI(r-) 
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1ms  bMn  ncslved.  That  it  can  prodvca  tha  output  slmulatlou  history  up  to 
tlma  t Is  fuarantaed  hy  tha  thxaa  raqulxemeiits  on  tha  modula.  That  It  will  is 
guarantaad  by  tha  corraet  modula  simulation  raqulramant.  In  order  for  the 
stmulatlon  module  to  realize  It  has  received  the  entire  input  simulation  history 
up  to  time  t it  may  require  packets  with  time  values  greater  than  or  equal  to  t, 
as  is  stated  in  the  condition  t s Tflnal(HSI).  The  simulation,  however,  will 
only  use  the  packets  with  time  values  lass  than  i in  eaimintiiig  the  output 
, values  with  time  values  less  than  or  equal  to  t. 


Proof  Lemma  1.2t 

Let  HI*  - data(HSI),  and  let  HO*  equal  the  output  history  of  the  actual 
module  when  it  starts  in  state  and  receives  the  input  history  HI*.  Then  by 
the  statement  of  the  lemma, 

HIU-s)  • data(HSI(r-s))  - HPU-s),  for  all  8>e. 

Hence,  by  the  three  requirements  for  the  actual  module 

HO’U)  - H0((). 

Furthermore,  by  the  correct  module  simulation  requirement,  if  t/inai  > 
TfinaMHSO).  then 

data(HS0((/lna/))  - H0*(f/lna/). 

By  the  statement  of  the  lemma,  t s tflnai,  therefore 

datalHSOU))  - H0*(r). 


Thus 


data(HSOm)  - H0*(r)  - HOU). 


ji 

' ! 
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This  lemma  will  allow  us  to  look  only  at  the  iaput  data  packets  with 
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time  velues  len  tlum  t,  wImb  tirioi  to  prove  the  correctnen  of  the  simulation 
up  to  and  including  time  t. 

Proof  of  Theorem  1. 

> 

The  main  theorem  will  be  proved  bjr  induction  on  the  sequence  of  time 
values 

• • • •tjt  • • • f 

where  tf  • 9,  and 

tf  < t£  < . . . < < ...  S 00, 

and  each  time  value  tj,  / > 0,  is  contained  in  some  actual  or  simulation  history 
for  the  system.  That  is.  Is  contained  In  one  of  the  following  histories:  I. 
the  system  in^t  history  to  the  actual  systemi  HOj,  the  output  history  of  some 
module  in  the  system  My;  SI,  the  ssrstem  Input  simulation  history;  or  HSOy,  the 
output  simulation  hlsiory  for  some  module  My.  As  mentioned  in  Chapter  2,  the 
history  and  simulation  history  for  any  port  must  be  a countable  sequence. 
Since  there  are  only  finitely  many  Input  and  output  ports  In  the  system,  only 
countably  many  time  values  can  appear  In  all  of  the  histories.  Thus,  the 
sequence  t0,t|....,f{,...  must  be  countable,  which  allows  us  to  perform 
induction  on  it. 

Induction  Hypothesis 

For  any  ty  c r9,ty,....t{,...,  such  that  s tfinalt 

a. )  data  (HSOy  (!{))  - H0y(t{),  for  all  modules  My,  and 

b. )  Either  ({  - 00,  or  for  any  output  port 

hso^(r|)  c hso^ 


I 

9 

i 

il 

i 

!j| 


] 

• 103  • 

That  ls»  the  stmulatlon  will  be  correct  throu^  time  and  all  output  ports  In 
the  simulation  will  produce  some  packet  with  time  value  greater  than  /{,  unless 

tj  • 00. 

Basist  / - 0. 

a. )  Initially,  HSOy(0)  - HOj(0)  - the  empty  history,  for  any  module  My. 

b. )  Initially,  HSIy(0)  - Hly(0)  - the  empty  history.  Hence,  Tfinal  (HSIy(0))  - 
0 for  any  module  My.  By  the  correct  coordination  requirement,  for  any  output 
port  Of.  of  module  My 

tlast-outj.  > Tf  Inal  (HSIyf0))  ■ 0. 

Thus,  hso^(0)  c hso^,  for  any  output  port  in  the  system. 

Induction:  Assume  true  for  I,  where  ti  < tflruU,  prove  true  for  Z-i-l.  < 

a.)  The  Monotlcity  of  Simulation  Output  Lemma  which  has  Just  been  proved 
will  be  applied  to  show  that  dataCHSOyOf^yi)  • HOyOy^y).  By  the  Induction 
assumption 

datadfSOyUy))  - HOyUy). 

for  all  modules  My  In  the  system.  Furthermore,  by  the  statement  of  the 
theorem. 


data(SI)  - I. 

Therefore,  since  all  communication  channels  in  the  simulation  operate  properly. 


data(HSIy(fy^y>8))  - HIyUy^y-8),  foT  all  8>0. 
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Next,  by  part  b).  of  the  Induction  assumption  hso,.(f{)  c hso,^  for  any 
output  port  Op,  In  the  simulation.  Then,  If  Input  port  la  connected  to  output 
port  Op^ 

- hso^df)  c hso^  - hsi^. 

Furthermore,  since  any  system  Input  port  will  receive  a packet  with  time 
greater  than  or  equal  to  tftnal,  and  tfinal  > t[, 

hsi^(/{)  c hsi^, 

for  any  system  Input  port  tj^.  Combining  these  two  facts, 

hsij^(f|)  c hsij^, 

for  any  Input  port,  In  the  system,  whether  It  Is  connected  to  another 
module,  or  It  Is  a system  Input  port.  No  packets  are  produced  In  the  simulation 
with  time  t such  that  ti  < t < hence 

c hsi^, 

for  any  input  port  In  the  system.  Therefore 

Tfinal  (HSIj)  i 

for  any  module  My.  Lmnma  1.2  can  therefore  be  applied  to  show  that 

data  (HSOy  ) ■ H0y(f|^y), 


for  any  module  My. 

b.)  As  has  Just  been  shown.  If  t'  - Tfinal  (HSIy)  for  the  module  My,  then  r’  a 
correct  coordination  requirement,  for  any  output  port  Op.  of  module 

My,  either 


or 


ttast-outp  > f’  a fj^y. 


1 


i 


1 
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That  la,  some  packet  with  time  value  greater  than  will  be  produced  on 
each  ovtput  port,  unless  - a>.  Thus,  for  any  output  port  d,.  In  the 
simulation,  either 


hso,.  c hso,^ 
or 

*!♦!  - 00. 

ThereforOt  by  Induction 

data(HSOj(ryina/))  - HO^dyina/), 
for  any  module  My  In  the  system. 

Finally,  to  show  that  the  module  My  would  have  the  same  final  state  In 
both  the  simulation  and  the  actual  system.  If  tflnal  - oo,  we  have  Just  shown 
that  data(HSO^(r/ina/))  - HOCr/lnol),  for  any  module  Mj^.  Furthermore,  for  the 
system  Input  ports,  the  statement  of  the  theorem  requires  that  data  (SI)  - I. 
Thus,  If  the  communication  links  between  simulation  modules  operate  correctly, 
and  tflnai  • oo 

data(HSIy)  - Hly, 

for  any  module  My.  By  the  statement  of  the  theorem.  My  Is  started  In  the  same 
Initial  state  Sg  In  both  the  simulation  and  the  actual  system,  therefore  by  the 
correct  module  simulation  requirement,  if  tflnai  - oo  and  the  simulation  module 
terminates,  then  both  the  simulation  module  and  the  actual  module  must  have 
the  same  final  state. 
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Appendix  2 

Correctness  of  the  Termination  Operations 

The  following  proof  shows  that  the  addition  of  the  termination  operations 
of  Chapter  4 to  the  simulation  modules  will  the  correctness  of  the 

simulation,  with  the  added  feature  that  the  simulation  will  terminate  once  the 
termination  conditions  are  satisfied. 


Theorem  2.  Correctness  of  Termination 


a.)  Suppose  a simulation  Is  performed  in  which  the  modules  to  he  simulated 
obey  the  three  requirements:  functionality  of  output,  monotonicity  of  output,  and 
finite  delay,  and  the  simulation  and  coordination  operations  of  each  simulation 
module  obey  the  three  requirements:  correct  module  simulation,  correct  orderinf 
of  output  packets,  and  correct  coordination,  and  furthermore  the  coordination 
operations  of  a simulation  module  cannot  cause  time  packets  (»)  to  be  sent  out 
by  the  simulation  module  unless 


lasn  **^"*jl’  “ "• 

Then  the  addition  of  termination  operations  to  the  simulation  modules  as 
described  in  Chapter  3 will  not  cause  any  of  these  requirements  to  be  violated. 


b.)  If  the  actual  system  ever  reaches  a state  In  which  no  modules  in  the 
system  will  ever  enter  the  firinc  mode  tinless  more  packets  are  received  on  the 
system  input  ports,  then  every  simulation  module  in  the  simulation  of  this 
system  will  eventually  produce  time  packets  with  value  oo  on  all  output  ports, 
if  all  system  input  ports  in  the  simulation  receive  time  packets  with  value  «. 


Proof  of  First  Port 

The  termination  operations  will  not  affect  the  actual  modules,  hence  the 
first  three  requirements  for  the  Correctness  of  Simulation  Theorem  will  hold. 
As  for  the  correct  module  simulation  requirement,  the  termination  operations  are 
designed  not  to  Interrupt  the  simulation  of  the  modules.  The  only  way  they 


could  potentially  cause  this  requirement  to  be  violated  would  be  by  termlnatln# 


tha  ftmulatlon  bafora  tha  tarmlAatlon  condltlona  ara  satUflad.  Furthamora, 
slnca  last  packats  contain  no  tima  values,  thalr  presence  will  not  affect  tha 
correct  ardariiif  of  output  packaU,  or  tha  correct  coordination  requlramants.  As 
lon^  as  tha  termination  operations  do  not  cause  the  simulation  modules  to  send 
out  time  packats  («)  before  the  termination  conditions  ara  satisfied,  neither  of 
these  last  two  requirements  will  be  violated  either. 

since  modules  can  communicate  with  each  other  only  in  the  form  of 
packets  sent  alond  the  data  channels,  the  conditions  for  termination  for  the 
modules  in  a connectivity  class  Cj  can  be  stated  asi 

1. )  For  each  simulation  module  c Cj  all  Input  ports  such 

that  I fromjeUtS|  have  received  time  packats  (oo). 

2. )  No  simulation  module  c Cy  can  simulate  the  firing  of  a 

module  without  recelvlnd  more  data  packets. 

3. )  No  simulation  module  In  Cy  will  ever  receive  further  data 

packets. 

For  a connectivity  class  which  contains  only  one  module  and  has  no 
self -loop,  there  are  no  termination  operations.  Thus,  as  long  as  the  termination 
operations  for  connectivity  classes  containing  cycles  do  not  cause  the  simulation 
modules  In  the  class  to  terminate  too  soon,  the  correctness  of  the  simulation 
will  be  maintained. 

Termination  operations  might  cause  the  simulation  modules  in  a class  to 
terminate  prematurely  In  one  of  two  ways.  First,  a test  of  the  class  might 
succeed,  even  though  the  termination  conditions  are  not  satisfied.  Second,  some 
simulation  module  might  receive  a time  packet  (»)  on  an  input  port  c 


fromjelass^,  before  any  test  has  succeeded,  and  then  proceed  to  send  out  time 
packets  (od)  from  all  output  ports,  even  though  the  termination  conditions  for 
the  class  are  not  satisfied.  This  second  case  can  be  ruled  out  rather  easily.  By 
the  further  restriction  which  has  been  placed  on  the  coordination  operations  in 
the  statement  of  the  theorem,  the  coordination  operations  cannot  cause  a 
simulation  module  c Cj  to  send  out  time  packets  (oo)  from  Its  output  ports, 
unless  time  packets  (oo)  have  been  received  on  all  Input  ports.  Including  those 
in  fromjelassf.  However,  no  simulation  module  t Cj  will  receive  a time 
packet  (oo)  on  an  Input  port  In  fromjclass^  unless  some  simulation  module 
c Cj  sends  a time  packet  (oo)  from  an  output  port  In  tojclassj.  Without 
any  termination  operations,  this  would  happen  only  If  ll{  had  already  receive  a 
time  packet  (oo)  on  all  Input  ports  Including  those  In  froiiuela»{.  Thus,  no 
simulation  module  can  be  the  first  simulation  module  In  the  class  to  send  time 
packets  (oo) . Therefore  the  coordination  operations  alone  cannot  cause  any 
simulation  modules  in  a class  to  terminate  If  the  class  contains  cycles. 
Furthermore,  the  termination  operations  cannot  causa  any  simulation  module  In 
a class  to  send  out  time  packets  (oo)  until  after  a test  has  succeeded. 

Thus,  the  proof  of  the  first  part  of  the  theorem  reduces  toi 


Lemma  2.1.  No  Premature  Termination 

Suppose  the  termination  control  module  T for  a connectivity  class  C j has 
received  time  packets  (oo)  on  all  input  ports  i froiiijelassj>,  and  no  firtog  of 
the  module  can  be  simulated  unless  more  data  packets  are  received.  If  T sends 
out  test  packets  (test. 4)  from  all  output  ports  c tojclassji  receives  K packets 
with  value,  test. 4,  in  return,  where 

((•142  (|tejelass||  - l)i 


and  it  receives  no  further  data  packets  while  waitiod  for  the  returning  test 
packets,  this  means  titat 

1. )  All  simulation  modules  c have  received  time  packets  (oo) 
on  all  input  ports  i frenuclassj. 

2. )  No  simulation  module  c Cy  can  simulate  the  firing  of  a 

module  without  receivind  more  data  packets. 

3. )  No  simulation  module  in  Cy  will  ever  receive  further  data 

packets. 


The  foUowlnd  sequence  of  assertions  proves  Lemma  2.1t 

1.)  If  every  simulation  module  c Cy  is  terminatahie,  meanind  that  it 
receives  a time  packet  (eo)  on  every  input  port  which  is  not  in  froiiijelasS|,  and 
it  eventually  stops  simulatliid  the  flrlnd  of  the  module,  then  durind  a test  (or 
reset)  of  the  class  Cy 

a. )  Each  simulation  module  in  Cy  will  receive  at  least  one  test 
(or  reset)  packet. 

b. )  Exactly  K test  (or  reset)  packets  will  be  created,  where 

K • 1 ♦ 2 (itojclass^l  - l). 

M^cCy 

c. )  At  least  one  test  (or  reset)  packet  will  be  received  on  each 
input  port  in  fremjclass^  for  every  c Cy. 

Assertion  la)  can  be  shown  by  induction  on  the  lendth  of  the  shortest 
path  from  T to  (there  must  be  a path  from  T to  any  other  module  in  a 
connectivity  class.)  As  a basis,  if  ( - 1,  then  T will  receive  a test 

(or  reset)  packet  shortly  after  T sends  out  test  (or  reset)  packets  from  each 
output  port  0^  < tojelanj>.  Now  assume  the  assertion  is  true  for  all  simulation 
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modules  In  the  class  with  a path  from  T of  landth  loss  than  or  equal  to  1. 
Then  If  there  is  a path  of  length  Z-i-l  from  T to  a simulation  module  there 
must  be  some  module  11^  c Cy,  such  that  -4  and  there  Is  a path  of 
lenith  I from  T to  Hence  the  Induction  assumption  applies  to  meanln# 
that  it  will  receive  at  least  one  test  packet.  As  IobA  as  la  tormlnatable.  It 
will  send  test  (or  reset)  packets  on  every  output  port  c tojclass^  Therefore, 
will  eventually  receive  a test  (or  reset)  packet. 

Assertion  lb)  follows  directly  llrom  la).  Initially,  T creates  and  sends  out 
|to_jclassy|  test  (or  reset)  packets.  The  first  time  some  other  simulation  module 
c Cy  receives  a test  (or  reset)  packet,  it  will  send  out  (tojelass^l  test  (or 
reset)  packets,  thereby  creatind  Itojclass^l  - 1 new  ones.  On  recelvlnd  any 
further  test  (or  reset)  packet,  a simulation  module  will  send  one  test  (or  reset) 
packet,  hence  no  new  test  packets  will  be  created,  nor  will  any  be  destroyed. 
By  assertion  la),  eventually  all  simulation  modules  will  receive  at  least  one  test 
(or  reset)  packet,  therefore  exactly  K test  (or  reset)  packets  will  be  created, 
where 

K • 1 ♦ 2 (itojelasr^l  - l). 

■l‘Cy 

Assertion  Ic)  also  follows  from  la).  Every  Input  port  1^^  In  fronuelast^  of 
a simulation  module  c Cy  is  connected  to  an  output  port  of  some  modulo 
M|  c Cy,  and  Is  In  tojclassj.  By  assertion  la),  will  receive  at  least  one 
test  (or  reset)  packet.  If  M|  Is  tormlnatable.  It  will  eventually  send  a test  (or 
reset)  packet  on  every  output  port  In  the  sot  fromjelasS|.  Therefore,  will 
afventually  receive  a tost  (or  reset)  packet  on  This  Is  true  for  any  input 
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port  la  fromjdani  of  ojr  itaulatton  modulo  ll(  c Cy. 

2. )  If  aomo  stmulatioa  modulo  la  not  tormlnaUblo,  than  loaa  than  K toat 
pockota  will  tw  croatod  during  a tost,  and  thoiafoia  tho  taat  cannot  auccaad. 

If  la  not  tarmlnataUo,  than  it  will  not  aand  out  any  taat  packata  ovun 
If  it  rocolvaa  any.  Thua  It  will  not  creata  Itojelan^l  - 1 toat  packata,  which 
maana  that  fowar  than  K taat  packata  will  be  created  during  a test  of  the  claaa. 
The  taat  cannot  auccaad  unless  T receives  K test  packets,  hence  the  test  cannot 
succeed  If  some  simulation  module  My  does  not  rocalva  time  packata  (»)  on  all 
Input  ports  which  are  not  In  fronuclass^,  or  It  does  not  stop  simulating  the 
firing  of  the  module. 

3. )  For  a tost  to  succeed,  no  simulation  module  can  racelvo  any  data  packets 
between  the  time  It  receives  Its  first  test  packet  and  tho  time  It  sends  Its  last 
tost  packet. 

If  a simulation  module  did  receive  a data  packet  during  this  time.  It  would 
send  out  at  least  one  packat  ftest.-).  Once  a (test.-)  packet  has  been  sent, 
the  test  must  fail,  because  any  termlnatabla  simulation  modulo  which  receives  a 
(test.-)  must  sand  out  a (test.-)  packat.  If  all  modules  are  tennlnatabla,  T 
will  receive  at  least  one  (test.-)  packet,  and  the  tost  will  fall.  If  some 
simulation  module  Is  not  termlnatabla,  tho  tost  will  fail  In  any  case. 
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4.)  If  a tost  succeeds,  no  simulation  module  c Cy  will  receive  any  data 
pockats  after  It  has  teoolvod  Its  last  tost  pocket. 
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Thlf  will  be  shown  by  contradiction.  SnppoM  a test  of  a class  succeeds, 
but  one  or  more  simulation  modules  receive  data  packets  after  receiving  their 
final  test  packets.  be  one  of  the  first  simulation  modules  for  which  this 

happens.  That  is,  during  the  test,  received  all  of  its  test  packets  and  later 
receives  a data  packet  p on  some  input  port  i^,  but  this  had  not  happened  to 
any  simulation  module  in  the  class  before  point.  If  is  not  in 
froinjclasS|,  then  could  not  have  sent  any  test  packets  before  receiving  this 
data  packet,  because  it  cannot  send  any  test  packets  before  receiving  a time 
packet  (eo)  on  Thus  if  a data  packet  la  received  on  an  input  port  which 
la  pot  In  froinjelasS|  after  any  test  packet  has  been  received  by  M^,  either  the 
simulation  module  would  not  be  terminatable,  or  would  send  out  a packet 
(test.-).  In  either  case,  the  test  would  fall.  Thus,  must  be  in  fromjclasS|, 
which,  by  assertion  Ic),  implies  that  a test  packet  was  received  on  input  port 
before  data  packet  p was  received.  By  the  flrst-in,  first-out  property  of  the 
oommunlcetlon  links  between  simulation  modules,  some  module  ll|  must  have 
sent  data  packet  p to  after  it  had  sent  a test  packet  to  This  possibUlty 
cen  be  eliminated  by  looking  at  two  casest 
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Case  1.  M|  - T 

The  termination  control  module  T did  not  send  out  any  test  packets  unless  it 
could  not  simulate  any  more  firings  without  receiving  more  data  packets.  Thus, 
in  order  for  T to  send  data  packet  p after  sending  test  packets,  it  must  receive 


6 - 

S.)  If  • test  succeeds,  then  no  slsiulatlon  module  in  tlie  class  can  ever  slmulato 
a firing  without  rocelvlns  more  data  packets,  nor  will  It  ever  recUve  more  data 
packets. 

If  a test  succeeds,  then  at  the  time  a simulation  module  sent  Its  first  test 
packet.  It  could  not  simulate  any  more  firings  without  receiving  more  data 
packets.  By  assertion  3),  the  simulation  module  did  not  receive  any  data 
packets  between  this  time  and  the  time  at  which  it  received  Its  last  test  packet. 
By  assertion  4),  the  simulation  module  did  not,  nor  will  It  receive  any  data 
packets  after  the  last  test  packet  was  received.  Therefore,  the  test  will  succeed 
oBly  If  all  simulation  modules  In  the  class  are  ready  to  be  terminated. 

This  completes  the  proof  that  the  addition  of  termination  operations  to  the 
simulation  modules  cannot  cause  them  to  terminate  too  soon.  Hence,  none  of 
the  six  requirements  for  the  Correctness  of  Simulation  Theorem  of  Appendix  1 
can  be  violated.  The  correctness  of  the  simulation  will  be  maintained. 

Proof  of  tho  Sooond  Part 

Proving  the  second  part  of  the  theorem  requires  showing  that  tho 
termination  operations  for  each  connectivity  class  will  cause  the  simulation 
modules  In  the  class  to  terminate,  once  tho  termination  conditions  for  the  class 
are  satlsflod.  If  a class  Cj  consists  of  a single  module  My  which  has  no 
anif'loop,  then  tho  correct  coordination  requirement  will  guarantee  that  time 
packets  (»)  will  be  sent  out  once  time  packets  with  value  od  have  been 
reeolved  on  all  Input  ports,  and  no  more  firings  at  the  modulo  osu  be  simulated. 
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Thiu,  tlilj  rl^  will  ftmlaato  obo*  tlw  tmnliiatlcm  ocmdltloBs  an  satlaflad. 
Fbr  connactlvltjr  rlarnu  cyclM,  it  must  be  Shown  that  once  the 

connectivity  class  rsachas  the  conditions  for  tannlnatlon,  any  prevlons  test  or 
reset  will  be  completad,  a new  test  of  the  class  will  be  Initiated,  and  this  test 
will  sncceed.  Those  refUlrements  ate  stated  In  the  following  lenunai 


i^irnim  2.2.  Eventual  Termination 

A. )  Completion  of  a Tost  or  Beset 

Suppose  the  termination  control  module  T for  a class  Cj  sends  a test  (or 
reset)  packet  from  each  output  port  in  iojAuMf.  If  every  simulation  module 
In  Cj  Is  termiJiaCab/e,  meaning  It  eventually  receives  time  packets  (ao)  on 
every  Input  port  which  Is  not  In  fromjclasS|,  and  It  eventually  stops 
the  flrlns  of  the  module,  then  all  simulation  modules  in  the  class 
wlU  receive  at  least  one  tast  (or  reset)  packet,  and  T will  eventually  receive  K 
test  (or  reset)  packets,  where 

K - 1 ♦ ^ (|tejelass||  - l). 

B. )  Eventual  Success  of  Test 

Suppose  every  simulation  module  In  reaches  a state  In  which  time 
packets  (oo)  have  been  received  on  all  input  ports  which  are  not  In  from_ctass^, 
no  firings  can  be  slmnlatad  without  racelvlns  more  dau  packets,  and  no  more 
data  packets  will  ever  be  received  by  M^.  Then  T will  send  out  test  packets 
(teat.-f)  from  all  output  ports  la  tojclassj-,  and  It  will  eventually  receive  K 
(test. 4)  packets  In  return  without  reoelvlnd  any  further  daU  packets. 

C. )  Termination  after  Successful  Tast 

If  T sends  out  time  packets  (oo)  on  all  of  Its  output  ports,  then  every 
simulation  module  la  the  class  will  eventually  receive  time  packets  (oo)  on 
all  Input  ports  and  hmice  will  terminate. 


The  foUowlnd  sequence  of  assertions  proves  each  part  of  Iiwnma  2.2i 
A.)  Completion  of  a Tbit  fit  BH. 
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1. )  If  9nmrt  liiiiulatloB  modula  la  tha  clan  la  tarmlaatabto,  than 

a.)  Each  simulation  modula  will  racalva  at  laast  ona  tast  (or 
raset)  packat. 

h.)  Exactly  K test  (or  raset)  packeta  will  ha  created. 

These  assertions  are  Identical  to  assertions  la)  and  lb)  la  the  proof  of 
Lemma  2.1. 

2. )  If  every  simulation  module  In  the  class  Cj  Is  tarmlnatabla,  T will  racalva  K 
tast  (or  raset)  packets. 

This  follows  from  the  way  In  which  the  sifnal  output  ports  ware  chosen. 
Every  simulation  module  except  f or  T has  a single  signal  output  port.  T has  no 
signal  output  port.  These  ports  are  chosen  In  suf^h  a way  that  If  wo  look  oaly 
at  the  simulation  modules  In  the  clan  and  the  channels  conaactad  to  their 
output  ports,  there  Is  a path  from  every  simulation  modula  to  T.  Thus,  the 
simulation  modules  and  the  channels  connected  to  the  signal  output  ports  fulfill 
the  necessary  requirements  for  a directed  tree  [1],  with  each  arc  pointing  from 
a son  to  Its  father.  That  Is 

1.  There  Is  a unique  root  node  (namely  T)  with  no  arcs  leavlag 
from  lti 

2.  Every  other  node  (K^  A T)  has  a single  arc  leaving  from  It 
(namely  the  channel  connected  to  the  signal  output  port)t  and 

3.  There  Is  a path  ffom  every  node  to  the  root  node. 

One  Important  property  of  trees  Is  that  they  are  acyclic,  hence  there  Is  no  path, 
Vfhlch  follows  oaly  signal  output  links.  During  the  tast  (or  reset) 
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opKtMxma,  K last  (ot  nwt)  ftduU  will  ba  ciaatad,  and  oaca  all  almulatlon 
aodnlai  ha«a  racaivad  at  laast  osa  taat  (or  raaat)  packat,  all  last  (or  resot) 
packats  will  seat  only  from  signal  output  ports.  These  packets  will  not  be 
dastroyad,  nor  can  any  termlnatable  simulation  module  hold  onto  them 
ladeflnltely,  hence  the  packets  can  only  be  propoiatad  toward  the  root  node  T. 
Tharafora  T will  evantually  rocalva  all  K test  (or  rosot)  packets,  and  the  test  (or 
raaat)  operations  will  be  completad. 

B.)  Eventual  Success  ^ Test. 

Suppose  every  simulation  module  in  a class  Cj  reaches  a state  in  which 
time  packets  (oo)  have  been  racaivad  on  all  input  ports  which  are  not  In 
frofluclass^,  no  firings  can  be  simulated  without  raoeivlad  mote  data  pockets, 
and  no  more  data  packets  will  ever  be  received  by  ll|. 

1.)  A new  tost  of  the  class  will  be  Initiated. 

If  the  simulation  modules  roach  the  abova-mentionad  state,  they  are  all 
termlnatable.  Hence,  by  part  A)  of  the  lemma,  any  previous  test  or  reset 
operations  will  be  completed.  Furthermore,  durlnd  the  reset  operations  every 
simulation  module  will  recalvo  a reset  packet.  Hence,  any  now  test  will  take 
place  as  If  no  previous  tests  had  occurred.  Furthermore,  once  the  reoat 
operations  are  completed,  a new  tost  will  bo  inltlatod. 

B.)  The  test  will  succeed. 


A 


raceives  its  first  test  packet  and  the  time  It  receives  Its  last  test  packet.  It  will 
send  out  (test.-f)  packets  as  lon^  as  It  receives  (test.+l  packets.  By  our 
assumption,  no  simulation  modules  will  receive  data  packets  once  the  test  has 
started.  Therefore,  since  T starts  the  test  by  sending  (teet.-f)  packets,  by  part 
A)  of  the  lemma,  K (test.-f)  will  be  created,  and  T will  eventually  receive  K 
(test.-f)  packets.  Thus,  the  test  will  succeed  once  the  termination  oonditlons 
for  the  class  are  satisfied. 

C.)  Termination  after  a Successful  Test. 

Suppose  the  test  of  a class  succeeds  and  T sends  time  packets  (a>)  from  all 
« 

output  ports. 

1. )  Every  simulation  module  In  Cj  will  receive  at  least  one  time  packet  (oo) 
OB  some  Input  port  In  fronuelass^. 

This  can  bo  shown  by  Induction  on  the  length  of  the  shortest  path  from  T 
to  If.  IB  fact,  the  proof  Is  virtually  Identical  to  the  proof  of  assertion  la)  In 
the  proof  of  Lemma  2.1. 

2. )  Every  slmulatton  module  c Cj  will  receive  time  packets  (co)  on  every 
Input  port. 

In  order  for  the  test  to  succeed,  must  have  received  time  packets  («) 
on  every  Input  port  which  is  not  In  fromjclsss^.  Furthermore,  by  assertion  1) 
any  module  ll|  c Cj  connected  to  must  receive  at  least  one  time  packet  («) 
on  some  input  port  « fromjclasS|.  Hence,  it  will  sand  out  time  packets  (oo) 
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OB  bU  output  porti,  ****^"**”2  obo  to  iBput  port  of  Buidulo  l|.  Thoroforo,  all 
modulos  In  Cj  will  xooeivo  tinw  packati  <oo)  ob  all  iBput  porta  onco 
the  tost  has  sucooedod. 

This  conplotas  the  proof  that  the  addition  of  the  tenninatlon  operations  to 
the  jrfmwiMion  modules  will  cause  the  simulatiou  to  terminate,  once  the 
tTTTiiwating  conditioBs  foT  tho  UTVIam  me  satisfied. 


J 


Official  Distribution  List 


Defense  Documentation  Center 
Cameron  Station 
Alexandria,  Va  22314 


Office  of  Naval  Research 
Information  Systems  Program 
Code  437 

Arlington,  Va  22217 


Office  of  Naval  Research 
Code  102IP 
Arlington,  Va  22217 


Office  of  Naval  Research 
Code  200 

Arlington,  Va  22217 


Office  of  Naval  Research 
Code  455 

Arlington,  Va  22217 


Office  of  Naval  Research 
Code  458 

Arlington,  Va  22217 


Office  of  Naval  Research 
Branch  Office,  Boston 
495  Summer  Street 
Boston,  Ma  02210 


Office  of  Naval  Research 
Branch  Office,  Chicago 
536  South  Clark  Street 
Chicago,  II  60605 


Office  of  Naval  Research 
Branch  Office,  Pasadena 
1030  East  Green  Street 
Pasadena,  Ca  91106 


New  York  Area  Office 
715  Broadway  - 5th  floor 

12  copies  New  York,  N.  Y.  10003  1 copy 


Naval  Research  Laboratory 
Technical  Information  Division 
Code  2627 

2 copies  Washington,  D.  C.  20375  6 copies 


Dr.  A.  L.  Slafkosky 
Scientific  Advisor 

6 copies  Commandant  of  the  Marine  Corps 

(Code  RD-1) 

Washington,  D.  C.  20380  1 copy 


1 copy  Naval  Electronics  Laboratory  Center 

Advanced  Software  Technology  Division 
Code  5200 

San  Diego,  Ca  92152  1 copy 

1 copy 

Mr.  E.  H.  Gleissner 

Naval  Ship  Research  & Development  Center 
Computation  & Mathematics  Department 
Bethesda,  Md  20084  1 copy 

1 copy 

Captain  Grace  M.  Hopper 
NAICOM/MIS  Planning  Branch  (OP-916D) 
Office  of  Chief  of  Naval  Operations 
Washington,  D.  C.  20350  1 copy 

1 copy 

Mr.  Kin  B.  Thompson 
Technical  Director 

Information  Systems  Division  (0P-91T) 
Office  of  Chief  of  Naval  Operations 
1 copy  Washington,  D.  C.  20350  1 copy 


1 copy 


